[GitHub] spark issue #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "spark.m...

2017-11-07 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/17436 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19634: [SPARK-22412][SQL] Fix incorrect comment in DataSourceSc...

2017-11-07 Thread vgankidi
Github user vgankidi commented on the issue: https://github.com/apache/spark/pull/19634 @gatorsmile I also wanted to discuss if we should consider other bin packing algorithms. According to this http://www.math.unl.edu/~s-sjessie1/203Handouts/Bin%20Packing.pdf, next fit decreasing

[GitHub] spark issue #19678: [SPARK-20646][core] Port executors page to new UI backen...

2017-11-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19678 **[Test build #83572 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83572/testReport)** for PR 19678 at commit

[GitHub] spark issue #19433: [SPARK-3162] [MLlib] Add local tree training for decisio...

2017-11-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19433 **[Test build #3983 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3983/testReport)** for PR 19433 at commit

[GitHub] spark issue #19687: [SPARK-19644][SQL]Clean up Scala reflection garbage afte...

2017-11-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19687 **[Test build #83571 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83571/testReport)** for PR 19687 at commit

[GitHub] spark pull request #19272: [Spark-21842][Mesos] Support Kerberos ticket rene...

2017-11-07 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/19272#discussion_r149549953 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -213,6 +216,14

[GitHub] spark issue #19433: [SPARK-3162] [MLlib] Add local tree training for decisio...

2017-11-07 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/19433 CC @dbtsai in case you're interested b/c of Sequoia forests --- - To unsubscribe, e-mail:

[GitHub] spark issue #19687: [SPARK-19644][SQL]Clean up Scala reflection garbage afte...

2017-11-07 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/19687 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19681: [SPARK-20652][sql] Store SQL UI data in the new app stat...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19681 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19687: [SPARK-19644][SQL]Clean up Scala reflection garba...

2017-11-07 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/19687 [SPARK-19644][SQL]Clean up Scala reflection garbage after creating Encoder ## What changes were proposed in this pull request? Because of the memory leak issue in

[GitHub] spark issue #19678: [SPARK-20646][core] Port executors page to new UI backen...

2017-11-07 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19678 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19681: [SPARK-20652][sql] Store SQL UI data in the new app stat...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19681 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83567/ Test FAILed. ---

[GitHub] spark issue #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark DataFra...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19459 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19678: [SPARK-20646][core] Port executors page to new UI backen...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19678 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark DataFra...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19459 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83569/ Test FAILed. ---

[GitHub] spark issue #19678: [SPARK-20646][core] Port executors page to new UI backen...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19678 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83568/ Test FAILed. ---

[GitHub] spark issue #19681: [SPARK-20652][sql] Store SQL UI data in the new app stat...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19681 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83570/ Test FAILed. ---

[GitHub] spark issue #19681: [SPARK-20652][sql] Store SQL UI data in the new app stat...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19681 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19663: [SPARK-22463][YARN][SQL][Hive]add hadoop/hive/hba...

2017-11-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19663#discussion_r149544676 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -687,6 +687,20 @@ private[spark] class Client(

[GitHub] spark pull request #19663: [SPARK-22463][YARN][SQL][Hive]add hadoop/hive/hba...

2017-11-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19663#discussion_r149544880 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -687,6 +687,20 @@ private[spark] class Client(

[GitHub] spark pull request #19663: [SPARK-22463][YARN][SQL][Hive]add hadoop/hive/hba...

2017-11-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19663#discussion_r149544716 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -687,6 +687,20 @@ private[spark] class Client(

[GitHub] spark issue #19682: [SPARK-22464] [SQL] No pushdown for Hive metastore parti...

2017-11-07 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/19682 Thanks for the fix! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19565: [SPARK-22111][MLLIB] OnlineLDAOptimizer should filter ou...

2017-11-07 Thread akopich
Github user akopich commented on the issue: https://github.com/apache/spark/pull/19565 ping @WeichenXu123 , @srowen , @hhbyyh Further comments? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #19681: [SPARK-20652][sql] Store SQL UI data in the new app stat...

2017-11-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19681 **[Test build #83570 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83570/testReport)** for PR 19681 at commit

[GitHub] spark issue #19657: [SPARK-22344][SPARKR] clean up install dir if running te...

2017-11-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19657 Will take a look within today. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #19681: [SPARK-20652][sql] Store SQL UI data in the new a...

2017-11-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19681#discussion_r149537039 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListener.scala --- @@ -0,0 +1,353 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #19525: [SPARK-22289] [ML] Add JSON support for Matrix pa...

2017-11-07 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/19525#discussion_r149522834 --- Diff: mllib-local/src/main/scala/org/apache/spark/ml/linalg/Matrices.scala --- @@ -827,6 +831,11 @@ class SparseMatrix @Since("2.0.0") (

[GitHub] spark pull request #19525: [SPARK-22289] [ML] Add JSON support for Matrix pa...

2017-11-07 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/19525#discussion_r149533876 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LogisticRegressionSuite.scala --- @@ -2769,6 +2769,20 @@ class

[GitHub] spark pull request #19525: [SPARK-22289] [ML] Add JSON support for Matrix pa...

2017-11-07 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/19525#discussion_r149522660 --- Diff: mllib-local/src/main/scala/org/apache/spark/ml/linalg/Matrices.scala --- @@ -476,6 +476,10 @@ class DenseMatrix @Since("2.0.0") (

[GitHub] spark pull request #19525: [SPARK-22289] [ML] Add JSON support for Matrix pa...

2017-11-07 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/19525#discussion_r149530436 --- Diff: mllib/src/main/scala/org/apache/spark/ml/linalg/JsonMatrixConverter.scala --- @@ -0,0 +1,79 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19525: [SPARK-22289] [ML] Add JSON support for Matrix pa...

2017-11-07 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/19525#discussion_r149532602 --- Diff: mllib/src/main/scala/org/apache/spark/ml/linalg/JsonMatrixConverter.scala --- @@ -0,0 +1,79 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19525: [SPARK-22289] [ML] Add JSON support for Matrix pa...

2017-11-07 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/19525#discussion_r149534129 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LogisticRegressionSuite.scala --- @@ -2769,6 +2769,20 @@ class

[GitHub] spark issue #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark DataFra...

2017-11-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19459 **[Test build #83569 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83569/testReport)** for PR 19459 at commit

[GitHub] spark pull request #19681: [SPARK-20652][sql] Store SQL UI data in the new a...

2017-11-07 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19681#discussion_r149529181 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListener.scala --- @@ -0,0 +1,353 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #19681: [SPARK-20652][sql] Store SQL UI data in the new a...

2017-11-07 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19681#discussion_r149530162 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListener.scala --- @@ -0,0 +1,353 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19678: [SPARK-20646][core] Port executors page to new UI backen...

2017-11-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19678 **[Test build #83568 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83568/testReport)** for PR 19678 at commit

[GitHub] spark issue #19678: [SPARK-20646][core] Port executors page to new UI backen...

2017-11-07 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19678 lgtm (assuming tests pass) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-11-07 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19250 > I think we can follow what Hive/Impala did for interoperability, i.e. create a config to interpret parquet INT96 as timezone-agnostic timestamp in parquet reader of Spark. If I understand

[GitHub] spark issue #19686: [MINOR][SQL] Fix a misleading `resultSchema` in ParquetF...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19686 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19686: [MINOR][SQL] Fix a misleading `resultSchema` in ParquetF...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19686 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83565/ Test PASSed. ---

[GitHub] spark issue #19672: [SPARK-22456][SQL] Add support for dayofweek function

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19672 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19672: [SPARK-22456][SQL] Add support for dayofweek function

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19672 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83564/ Test PASSed. ---

[GitHub] spark issue #19686: [MINOR][SQL] Fix a misleading `resultSchema` in ParquetF...

2017-11-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19686 **[Test build #83565 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83565/testReport)** for PR 19686 at commit

[GitHub] spark issue #19672: [SPARK-22456][SQL] Add support for dayofweek function

2017-11-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19672 **[Test build #83564 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83564/testReport)** for PR 19672 at commit

[GitHub] spark pull request #19677: [SPARK-20645][core] Port environment page to new ...

2017-11-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19677 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19677: [SPARK-20645][core] Port environment page to new UI back...

2017-11-07 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19677 merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-11-07 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19250 Ah now I understand this issue. Yes Spark doesn't follow the SQL standard, the Spark timestamp is actually TIMESTAMP WITH LOCAL TIME ZONE, which is not SQL standard but used in some databases

[GitHub] spark issue #19686: [MINOR][SQL] Fix a misleading `resultSchema` in ParquetF...

2017-11-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19686 Thank you for review. Yes. Technically, the previous one is not wrong. It means an unordered set of attributes. So it's minor. In both

[GitHub] spark issue #19681: [SPARK-20652][sql] Store SQL UI data in the new app stat...

2017-11-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19681 **[Test build #83567 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83567/testReport)** for PR 19681 at commit

[GitHub] spark issue #19683: [SPARK-21657][SQL] optimize explode quadratic memory con...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19683 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83566/ Test FAILed. ---

[GitHub] spark issue #19683: [SPARK-21657][SQL] optimize explode quadratic memory con...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19683 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19683: [SPARK-21657][SQL] optimize explode quadratic memory con...

2017-11-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19683 **[Test build #83566 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83566/testReport)** for PR 19683 at commit

[GitHub] spark issue #19681: [SPARK-20652][sql] Store SQL UI data in the new app stat...

2017-11-07 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19681 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19646: [SPARK-22417][PYTHON] Fix for createDataFrame from panda...

2017-11-07 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/19646 Thanks @cloud-fan , I'll open up a PR for the backport to 2.2 --- - To unsubscribe, e-mail:

[GitHub] spark issue #19631: [SPARK-22372][core, yarn] Make cluster submission use Sp...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19631 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19631: [SPARK-22372][core, yarn] Make cluster submission use Sp...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19631 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83555/ Test PASSed. ---

[GitHub] spark issue #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "spark.m...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17436 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83561/ Test FAILed. ---

[GitHub] spark issue #19631: [SPARK-22372][core, yarn] Make cluster submission use Sp...

2017-11-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19631 **[Test build #83555 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83555/testReport)** for PR 19631 at commit

[GitHub] spark issue #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "spark.m...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17436 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "spark.m...

2017-11-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17436 **[Test build #83561 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83561/testReport)** for PR 17436 at commit

[GitHub] spark issue #19678: [SPARK-20646][core] Port executors page to new UI backen...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19678 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19678: [SPARK-20646][core] Port executors page to new UI backen...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19678 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83556/ Test PASSed. ---

[GitHub] spark issue #19678: [SPARK-20646][core] Port executors page to new UI backen...

2017-11-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19678 **[Test build #83556 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83556/testReport)** for PR 19678 at commit

[GitHub] spark issue #19679: [SPARK-20647][core] Port StorageTab to the new UI backen...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19679 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19679: [SPARK-20647][core] Port StorageTab to the new UI backen...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19679 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83560/ Test FAILed. ---

[GitHub] spark issue #19679: [SPARK-20647][core] Port StorageTab to the new UI backen...

2017-11-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19679 **[Test build #83560 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83560/testReport)** for PR 19679 at commit

[GitHub] spark issue #19681: [SPARK-20652][sql] Store SQL UI data in the new app stat...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19681 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83563/ Test FAILed. ---

[GitHub] spark issue #19681: [SPARK-20652][sql] Store SQL UI data in the new app stat...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19681 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19681: [SPARK-20652][sql] Store SQL UI data in the new app stat...

2017-11-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19681 **[Test build #83563 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83563/testReport)** for PR 19681 at commit

[GitHub] spark issue #19479: [SPARK-17074] [SQL] Generate equi-height histogram in co...

2017-11-07 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19479 LGTM overall. We should also consider how to show the histogram in ANALYZE COLUMN, for debug purpose. --- - To unsubscribe,

[GitHub] spark pull request #19479: [SPARK-17074] [SQL] Generate equi-height histogra...

2017-11-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19479#discussion_r149504772 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/Statistics.scala --- @@ -220,29 +239,46 @@ object ColumnStat extends

[GitHub] spark pull request #19479: [SPARK-17074] [SQL] Generate equi-height histogra...

2017-11-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19479#discussion_r149503350 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/Statistics.scala --- @@ -275,6 +317,98 @@ object ColumnStat extends

[GitHub] spark pull request #19682: [SPARK-22464] [SQL] No pushdown for Hive metastor...

2017-11-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19682 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19682: [SPARK-22464] [SQL] No pushdown for Hive metastore parti...

2017-11-07 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19682 can you send a new PR for 2.2? thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19682: [SPARK-22464] [SQL] No pushdown for Hive metastore parti...

2017-11-07 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19682 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19681: [SPARK-20652][sql] Store SQL UI data in the new a...

2017-11-07 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19681#discussion_r149500473 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListener.scala --- @@ -0,0 +1,353 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19646: [SPARK-22417][PYTHON] Fix for createDataFrame from panda...

2017-11-07 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19646 also backport to 2.2 since we consider it's a bug. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #19646: [SPARK-22417][PYTHON] Fix for createDataFrame fro...

2017-11-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19646 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19646: [SPARK-22417][PYTHON] Fix for createDataFrame from panda...

2017-11-07 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19646 LGTM, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19672: [SPARK-22456][SQL] Add support for dayofweek func...

2017-11-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19672#discussion_r149493317 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala --- @@ -376,6 +376,7 @@ object FunctionRegistry {

[GitHub] spark pull request #19646: [SPARK-22417][PYTHON] Fix for createDataFrame fro...

2017-11-07 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19646#discussion_r149491411 --- Diff: python/pyspark/sql/session.py --- @@ -416,6 +417,50 @@ def _createFromLocal(self, data, schema): data = [schema.toInternal(row)

[GitHub] spark pull request #19646: [SPARK-22417][PYTHON] Fix for createDataFrame fro...

2017-11-07 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19646#discussion_r149490465 --- Diff: python/pyspark/sql/session.py --- @@ -416,6 +417,50 @@ def _createFromLocal(self, data, schema): data = [schema.toInternal(row)

[GitHub] spark issue #19657: [SPARK-22344][SPARKR] clean up install dir if running te...

2017-11-07 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/19657 AppVeyor still has an error ``` 1. Failure: traverseParentDirs (@test_utils.R#252) - `dirs` not equal to `expect`. 1/4 mismatches x[1]:

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r149485899 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,192 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r149485321 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,192 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r149486527 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,192 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r149486897 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,192 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark issue #19640: [SPARK-16986][CORE][WEB-UI] Support configure history se...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19640 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83550/ Test PASSed. ---

[GitHub] spark issue #19640: [SPARK-16986][CORE][WEB-UI] Support configure history se...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19640 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19640: [SPARK-16986][CORE][WEB-UI] Support configure history se...

2017-11-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19640 **[Test build #83550 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83550/testReport)** for PR 19640 at commit

[GitHub] spark issue #19686: [MINOR][SQL] Fix a misleading `resultSchema` in ParquetF...

2017-11-07 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19686 Why is the order important here? Wasn't obvious to mr --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19682: [SPARK-22464] [SQL] No pushdown for Hive metastore parti...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19682 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83552/ Test PASSed. ---

[GitHub] spark issue #19682: [SPARK-22464] [SQL] No pushdown for Hive metastore parti...

2017-11-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19682 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19682: [SPARK-22464] [SQL] No pushdown for Hive metastore parti...

2017-11-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19682 **[Test build #83552 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83552/testReport)** for PR 19682 at commit

[GitHub] spark pull request #19659: [SPARK-19668][ML] Multiple NGram sizes

2017-11-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/19659#discussion_r149480386 --- Diff: examples/src/main/python/ml/n_gram_example.py --- @@ -35,7 +35,7 @@ (2, ["Logistic", "regression", "models", "are", "neat"])

[GitHub] spark pull request #19659: [SPARK-19668][ML] Multiple NGram sizes

2017-11-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/19659#discussion_r149478967 --- Diff: mllib/src/main/scala/org/apache/spark/ml/extensions/seq/package.scala --- @@ -0,0 +1,121 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19659: [SPARK-19668][ML] Multiple NGram sizes

2017-11-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/19659#discussion_r149480525 --- Diff: mllib/src/main/scala/org/apache/spark/ml/extensions/seq/package.scala --- @@ -0,0 +1,121 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19659: [SPARK-19668][ML] Multiple NGram sizes

2017-11-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/19659#discussion_r149480876 --- Diff: mllib/src/main/scala/org/apache/spark/ml/extensions/seq/package.scala --- @@ -0,0 +1,121 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19659: [SPARK-19668][ML] Multiple NGram sizes

2017-11-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/19659#discussion_r149479380 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/NGram.scala --- @@ -42,11 +42,22 @@ class NGram @Since("1.5.0") (@Since("1.5.0") override val

[GitHub] spark pull request #19659: [SPARK-19668][ML] Multiple NGram sizes

2017-11-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/19659#discussion_r149479801 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/NGramSuite.scala --- @@ -76,11 +76,32 @@ class NGramSuite extends SparkFunSuite with

<    1   2   3   4   5   >