[GitHub] spark issue #20350: [SPARK-23179][SQL] Support option to throw exception if ...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20350 **[Test build #93401 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93401/testReport)** for PR 20350 at commit

[GitHub] spark issue #20350: [SPARK-23179][SQL] Support option to throw exception if ...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20350 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20350: [SPARK-23179][SQL] Support option to throw exception if ...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20350 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93401/ Test PASSed. ---

[GitHub] spark issue #21835: [SPARK-24779]Add sequence / map_concat / map_from_entrie...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21835 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93402/ Test PASSed. ---

[GitHub] spark pull request #21584: [SPARK-24433][K8S] Initial R Bindings for SparkR ...

2018-07-22 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21584#discussion_r204229713 --- Diff: resource-managers/kubernetes/docker/src/main/dockerfiles/spark/bindings/R/Dockerfile --- @@ -0,0 +1,29 @@ +# +# Licensed to the

[GitHub] spark pull request #21584: [SPARK-24433][K8S] Initial R Bindings for SparkR ...

2018-07-22 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21584#discussion_r204229663 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesDriverBuilder.scala --- @@ -71,8 +75,9 @@

[GitHub] spark pull request #21584: [SPARK-24433][K8S] Initial R Bindings for SparkR ...

2018-07-22 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21584#discussion_r204229514 --- Diff: bin/docker-image-tool.sh --- @@ -79,11 +80,16 @@ function build { docker build $NOCACHEARG "${BINDING_BUILD_ARGS[@]}" \ -t

[GitHub] spark pull request #21584: [SPARK-24433][K8S] Initial R Bindings for SparkR ...

2018-07-22 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21584#discussion_r204229518 --- Diff: bin/docker-image-tool.sh --- @@ -79,11 +80,16 @@ function build { docker build $NOCACHEARG "${BINDING_BUILD_ARGS[@]}" \ -t

[GitHub] spark issue #21584: [SPARK-24433][K8S] Initial R Bindings for SparkR on K8s

2018-07-22 Thread ifilonenko
Github user ifilonenko commented on the issue: https://github.com/apache/spark/pull/21584 @felixcheung the PRB isn’t currently configured with capability to build —r distributions. Waiting on @shaneknapp, which is why it is unable to build the image. With an R supported

[GitHub] spark pull request #21439: [SPARK-24391][SQL] Support arrays of any types by...

2018-07-22 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21439#discussion_r204231416 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala --- @@ -61,6 +61,7 @@ class JacksonParser( dt match

[GitHub] spark pull request #21439: [SPARK-24391][SQL] Support arrays of any types by...

2018-07-22 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21439#discussion_r204232671 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala --- @@ -61,6 +61,7 @@ class JacksonParser( dt match

[GitHub] spark issue #21822: [SPARK-24865] Remove AnalysisBarrier - WIP

2018-07-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21822 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21837: [SPARK-24881][SQL] New Avro options - compression and co...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21837 **[Test build #93404 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93404/testReport)** for PR 21837 at commit

[GitHub] spark issue #21775: [SPARK-24812][SQL] Last Access Time in the table descrip...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21775 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21775: [SPARK-24812][SQL] Last Access Time in the table descrip...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21775 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93407/ Test FAILed. ---

[GitHub] spark issue #21775: [SPARK-24812][SQL] Last Access Time in the table descrip...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21775 **[Test build #93407 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93407/testReport)** for PR 21775 at commit

[GitHub] spark issue #21699: [SPARK-24722][SQL] pivot() with Column type argument

2018-07-22 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/21699 @gatorsmile The PR https://github.com/apache/spark/pull/21753 has been merged already. Can we continue with this PR? @maryannxue I added the tests you asked for in

[GitHub] spark issue #21838: [SPARK-24811][SQL]Avro: add new function from_avro and t...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21838 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21838: [SPARK-24811][SQL]Avro: add new function from_avro and t...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21838 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21758: [SPARK-24795][CORE] Implement barrier execution m...

2018-07-22 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21758#discussion_r204236692 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -359,20 +366,55 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark issue #21775: [SPARK-24812][SQL] Last Access Time in the table descrip...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21775 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93406/ Test FAILed. ---

[GitHub] spark issue #21775: [SPARK-24812][SQL] Last Access Time in the table descrip...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21775 **[Test build #93406 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93406/testReport)** for PR 21775 at commit

[GitHub] spark issue #21775: [SPARK-24812][SQL] Last Access Time in the table descrip...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21775 **[Test build #93406 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93406/testReport)** for PR 21775 at commit

[GitHub] spark issue #21775: [SPARK-24812][SQL] Last Access Time in the table descrip...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21775 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21838: [SPARK-24811][SQL]Avro: add new function from_avr...

2018-07-22 Thread gengliangwang
GitHub user gengliangwang opened a pull request: https://github.com/apache/spark/pull/21838 [SPARK-24811][SQL]Avro: add new function from_avro and to_avro ## What changes were proposed in this pull request? 1. Add a new function from_avro for parsing a binary column of avro

[GitHub] spark issue #21775: [SPARK-24812][SQL] Last Access Time in the table descrip...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21775 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21840: Initial implementation of copy function

2018-07-22 Thread ssimeonov
Github user ssimeonov commented on a diff in the pull request: https://github.com/apache/spark/pull/21840#discussion_r204244143 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3858,3 +3858,29 @@ object ArrayUnion

[GitHub] spark pull request #21840: Initial implementation of copy function

2018-07-22 Thread ssimeonov
Github user ssimeonov commented on a diff in the pull request: https://github.com/apache/spark/pull/21840#discussion_r204243887 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3858,3 +3858,29 @@ object ArrayUnion

[GitHub] spark pull request #21840: Initial implementation of copy function

2018-07-22 Thread ssimeonov
Github user ssimeonov commented on a diff in the pull request: https://github.com/apache/spark/pull/21840#discussion_r204245778 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Column.scala --- @@ -1234,6 +1234,8 @@ class Column(val expr: Expression) extends Logging {

[GitHub] spark pull request #21840: Initial implementation of copy function

2018-07-22 Thread ssimeonov
Github user ssimeonov commented on a diff in the pull request: https://github.com/apache/spark/pull/21840#discussion_r204244400 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3858,3 +3858,29 @@ object ArrayUnion

[GitHub] spark issue #21838: [SPARK-24811][SQL]Avro: add new function from_avro and t...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21838 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93410/ Test PASSed. ---

[GitHub] spark issue #21838: [SPARK-24811][SQL]Avro: add new function from_avro and t...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21838 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19492: [SPARK-22228][SQL] Add support for array...

2018-07-22 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/19492 Closing as we have a newer PR (https://github.com/apache/spark/pull/21439) which uses the refactored classes (which made the change easier compared to this PR). Thanks. ---

[GitHub] spark pull request #21840: [WIP] New copy() method for Column of StructType

2018-07-22 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21840#discussion_r204249828 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3858,3 +3858,29 @@ object ArrayUnion {

[GitHub] spark pull request #21840: [WIP] New copy() method for Column of StructType

2018-07-22 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21840#discussion_r204250170 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3858,3 +3858,29 @@ object ArrayUnion {

[GitHub] spark pull request #21439: [SPARK-24391][SQL] Support arrays of any types by...

2018-07-22 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21439#discussion_r204250446 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala --- @@ -101,6 +102,17 @@ class JacksonParser( }

[GitHub] spark pull request #21841: [SPARK-24768][SQL]Avro: remove implicit class Avr...

2018-07-22 Thread gengliangwang
GitHub user gengliangwang opened a pull request: https://github.com/apache/spark/pull/21841 [SPARK-24768][SQL]Avro: remove implicit class AvroDataFrameWriter/AvroDataFrameReader ## What changes were proposed in this pull request? As per Reynold's comment:

[GitHub] spark issue #21839: [SPARK-24339][SQL] Prunes the unused columns from child ...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21839 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21839: [SPARK-24339][SQL] Prunes the unused columns from child ...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21839 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93413/ Test PASSed. ---

[GitHub] spark issue #21841: [SPARK-24768][SQL]Avro: remove implicit class AvroDataFr...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21841 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21758 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21837: [SPARK-24881][SQL] New Avro options - compression...

2018-07-22 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21837#discussion_r204252552 --- Diff: external/avro/src/main/scala/org/apache/spark/sql/avro/AvroFileFormat.scala --- @@ -133,8 +134,7 @@ private[avro] class AvroFileFormat extends

[GitHub] spark issue #21754: [SPARK-24705][SQL] Cannot reuse an exchange operator wit...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21754 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93415/ Test PASSed. ---

[GitHub] spark issue #21754: [SPARK-24705][SQL] Cannot reuse an exchange operator wit...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21754 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21758 **[Test build #93411 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93411/testReport)** for PR 21758 at commit

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21758 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21758 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21758 **[Test build #93412 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93412/testReport)** for PR 21758 at commit

[GitHub] spark issue #17461: [SPARK-20082][ml] LDA incremental model learning

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17461 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21754: [SPARK-24705][SQL] Cannot reuse an exchange operator wit...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21754 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21754: [SPARK-24705][SQL] Cannot reuse an exchange operator wit...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21754 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #21802: [SPARK-23928][SQL] Add shuffle collection functio...

2018-07-22 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21802#discussion_r204249027 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CollectionExpressionsSuite.scala --- @@ -1419,4 +1421,71 @@ class

[GitHub] spark issue #21841: [SPARK-24768][SQL]Avro: remove implicit class AvroDataFr...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21841 **[Test build #93417 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93417/testReport)** for PR 21841 at commit

[GitHub] spark issue #21841: [SPARK-24768][SQL]Avro: remove implicit class AvroDataFr...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21841 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21758 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21758 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93412/ Test PASSed. ---

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21758 **[Test build #93412 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93412/testReport)** for PR 21758 at commit

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21758 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21758 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21821: [SPARK-24867] [SQL] Add AnalysisBarrier to DataFrameWrit...

2018-07-22 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21821 shall we fix the non-idempotent analyzer rule for 2.3.2? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #21839: [SPARK-24339][SQL] Prunes the unused columns from child ...

2018-07-22 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/21839 @gatorsmile @maropu This is the follow up PR for #21447, please have a look when you have time, thanks. --- - To

[GitHub] spark pull request #21839: [SPARK-24339][SQL] Prunes the unused columns from...

2018-07-22 Thread xuanyuanking
GitHub user xuanyuanking opened a pull request: https://github.com/apache/spark/pull/21839 [SPARK-24339][SQL] Prunes the unused columns from child of ScriptTransformation ## What changes were proposed in this pull request? Modify the strategy in ColumnPruning to add a

[GitHub] spark issue #21775: [SPARK-24812][SQL] Last Access Time in the table descrip...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21775 **[Test build #93409 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93409/testReport)** for PR 21775 at commit

[GitHub] spark issue #21775: [SPARK-24812][SQL] Last Access Time in the table descrip...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21775 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93409/ Test PASSed. ---

[GitHub] spark issue #21754: [SPARK-24705][SQL] Cannot reuse an exchange operator wit...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21754 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19492: [SPARK-22228][SQL] Add support for array

2018-07-22 Thread mgaido91
Github user mgaido91 closed the pull request at: https://github.com/apache/spark/pull/19492 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21811: [SPARK-24801][CORE] Avoid memory waste by empty byte[] a...

2018-07-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21811 Does it make sense to release `byteChannel` at `deallocate()`? ``` if (byteChannel != null) { byteChannel = null; } ``` ---

[GitHub] spark pull request #21830: [SPARK-24878][SQL] Fix reverse function for array...

2018-07-22 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21830#discussion_r204250861 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1244,46 +1244,50 @@ case class

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21758 **[Test build #93411 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93411/testReport)** for PR 21758 at commit

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21758 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93411/ Test PASSed. ---

[GitHub] spark issue #21842: [Minor][ML]Added UT for checking maximum number of featu...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21842 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21754: [SPARK-24705][SQL] Cannot reuse an exchange operator wit...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21754 **[Test build #93415 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93415/testReport)** for PR 21754 at commit

[GitHub] spark pull request #20861: [SPARK-23599][SQL] Use RandomUUIDGenerator in Uui...

2018-07-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20861#discussion_r204242507 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1994,6 +1996,20 @@ class Analyzer( }

[GitHub] spark pull request #21840: Initial implementation of copy function

2018-07-22 Thread MaxGekk
GitHub user MaxGekk opened a pull request: https://github.com/apache/spark/pull/21840 Initial implementation of copy function ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested?

[GitHub] spark issue #21123: [SPARK-24045][SQL]Create base class for file data source...

2018-07-22 Thread tengpeng
Github user tengpeng commented on the issue: https://github.com/apache/spark/pull/21123 Any updates on this PR? Yes, I know this is a temporary hack, but without it being merged, it is not possible to migrate other data sources to V2 (experimentally). ---

[GitHub] spark pull request #21840: Initial implementation of copy function

2018-07-22 Thread MaxGekk
Github user MaxGekk closed the pull request at: https://github.com/apache/spark/pull/21840 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21837: [SPARK-24881][SQL] New Avro options - compression...

2018-07-22 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21837#discussion_r204243507 --- Diff: external/avro/src/test/scala/org/apache/spark/sql/avro/AvroSuite.scala --- @@ -889,4 +889,30 @@ class AvroSuite extends QueryTest with

[GitHub] spark issue #21699: [SPARK-24722][SQL] pivot() with Column type argument

2018-07-22 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/21699 > it's basically reverting Reynold's decision. is he okay with it? As far as I know, yes. /cc @rxin --- - To

[GitHub] spark issue #21754: [SPARK-24705][SQL] Cannot reuse an exchange operator wit...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21754 **[Test build #93414 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93414/testReport)** for PR 21754 at commit

[GitHub] spark issue #21754: [SPARK-24705][SQL] Cannot reuse an exchange operator wit...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21754 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21754: [SPARK-24705][SQL] Cannot reuse an exchange operator wit...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21754 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21754: [SPARK-24705][SQL] Cannot reuse an exchange operator wit...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21754 **[Test build #93415 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93415/testReport)** for PR 21754 at commit

[GitHub] spark issue #21123: [SPARK-24045][SQL]Create base class for file data source...

2018-07-22 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/21123 @cloud-fan is going to update the data source V2 read path: https://issues.apache.org/jira/browse/SPARK-24882 I will update the file source base on it . ---

[GitHub] spark issue #21839: [SPARK-24339][SQL] Prunes the unused columns from child ...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21839 **[Test build #93413 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93413/testReport)** for PR 21839 at commit

[GitHub] spark issue #21842: [Minor][ML]Added UT for checking maximum number of featu...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21842 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21842: [Minor][ML]Added UT for checking maximum number o...

2018-07-22 Thread shahidki31
GitHub user shahidki31 opened a pull request: https://github.com/apache/spark/pull/21842 [Minor][ML]Added UT for checking maximum number of features for GeneralizedLinearRegression and WeightedLeastSquares Currently in the GeneralizedLinearRegression and WeightedLeastSquare

[GitHub] spark issue #21842: [Minor][ML]Added UT for checking maximum number of featu...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21842 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21839: [SPARK-24339][SQL] Prunes the unused columns from child ...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21839 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21839: [SPARK-24339][SQL] Prunes the unused columns from child ...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21839 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21839: [SPARK-24339][SQL] Prunes the unused columns from child ...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21839 **[Test build #93413 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93413/testReport)** for PR 21839 at commit

[GitHub] spark issue #21699: [SPARK-24722][SQL] pivot() with Column type argument

2018-07-22 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21699 it's basically reverting Reynold's decision. is he okay with it? --- - To unsubscribe, e-mail:

[GitHub] spark issue #21837: [SPARK-24881][SQL] New Avro options - compression and co...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21837 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93408/ Test PASSed. ---

[GitHub] spark issue #21837: [SPARK-24881][SQL] New Avro options - compression and co...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21837 **[Test build #93408 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93408/testReport)** for PR 21837 at commit

[GitHub] spark issue #21837: [SPARK-24881][SQL] New Avro options - compression and co...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21837 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21840: Initial implementation of copy function

2018-07-22 Thread ssimeonov
Github user ssimeonov commented on a diff in the pull request: https://github.com/apache/spark/pull/21840#discussion_r204246252 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Column.scala --- @@ -1234,6 +1234,8 @@ class Column(val expr: Expression) extends Logging {

[GitHub] spark issue #21754: [SPARK-24705][SQL] Cannot reuse an exchange operator wit...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21754 **[Test build #93414 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93414/testReport)** for PR 21754 at commit

[GitHub] spark issue #21754: [SPARK-24705][SQL] Cannot reuse an exchange operator wit...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21754 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93414/ Test FAILed. ---

[GitHub] spark issue #21838: [SPARK-24811][SQL]Avro: add new function from_avro and t...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21838 **[Test build #93410 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93410/testReport)** for PR 21838 at commit

[GitHub] spark issue #19222: [SPARK-10399][SPARK-23879][CORE][SQL] Introduce multiple...

2018-07-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19222 ping @rednaxelafx --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21821: [SPARK-24867] [SQL] Add AnalysisBarrier to DataFrameWrit...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21821 Test PASSed. Refer to this link for build results (access rights to CI server needed):

  1   2   3   >