[GitHub] spark issue #19718: [SPARK-19644][SQL] Clean up Scala reflection garbage aft...

2017-11-10 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/19718 Thanks! Merging to 2.2. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revie

[GitHub] spark pull request #19718: [SPARK-19644][SQL] Clean up Scala reflection garb...

2017-11-10 Thread zsxwing
Github user zsxwing closed the pull request at: https://github.com/apache/spark/pull/19718 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19630: wip: [SPARK-22409] Introduce function type argument in p...

2017-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19630 **[Test build #83701 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83701/testReport)** for PR 19630 at commit [`75d31b4`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #19630: wip: [SPARK-22409] Introduce function type argument in p...

2017-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19630 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83701/ Test FAILed. ---

[GitHub] spark issue #19630: wip: [SPARK-22409] Introduce function type argument in p...

2017-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19630 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19719: [SPARK-22487][SQL][followup] still keep spark.sql.hive.v...

2017-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19719 **[Test build #83708 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83708/testReport)** for PR 19719 at commit [`0ce241d`](https://github.com/apache/spark/commit/0c

[GitHub] spark issue #19630: wip: [SPARK-22409] Introduce function type argument in p...

2017-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19630 **[Test build #83709 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83709/testReport)** for PR 19630 at commit [`94cded6`](https://github.com/apache/spark/commit/94

[GitHub] spark pull request #19433: [SPARK-3162] [MLlib] Add local tree training for ...

2017-11-10 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19433#discussion_r150352747 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/model/InformationGainStats.scala --- @@ -112,7 +113,7 @@ private[spark] object ImpurityStats {

[GitHub] spark pull request #19433: [SPARK-3162] [MLlib] Add local tree training for ...

2017-11-10 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19433#discussion_r150154413 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/AggUpdateUtils.scala --- @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache Software Fou

[GitHub] spark pull request #19433: [SPARK-3162] [MLlib] Add local tree training for ...

2017-11-10 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19433#discussion_r150309552 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/SplitUtils.scala --- @@ -0,0 +1,215 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] spark pull request #19433: [SPARK-3162] [MLlib] Add local tree training for ...

2017-11-10 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19433#discussion_r150160368 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/SplitUtils.scala --- @@ -0,0 +1,215 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] spark pull request #19433: [SPARK-3162] [MLlib] Add local tree training for ...

2017-11-10 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19433#discussion_r150159113 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/SplitUtils.scala --- @@ -0,0 +1,215 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] spark pull request #19433: [SPARK-3162] [MLlib] Add local tree training for ...

2017-11-10 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19433#discussion_r150159513 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/SplitUtils.scala --- @@ -0,0 +1,215 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] spark pull request #19433: [SPARK-3162] [MLlib] Add local tree training for ...

2017-11-10 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19433#discussion_r150158027 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/RandomForest.scala --- @@ -627,221 +621,37 @@ private[spark] object RandomForest extends Logg

[GitHub] spark pull request #19433: [SPARK-3162] [MLlib] Add local tree training for ...

2017-11-10 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19433#discussion_r150349566 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/RandomForest.scala --- @@ -852,6 +662,41 @@ private[spark] object RandomForest extends Loggin

[GitHub] spark issue #19720: [SPARK-22494][SQL] Fix 64KB limit exception with Coalesc...

2017-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19720 **[Test build #83707 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83707/testReport)** for PR 19720 at commit [`f0edd7e`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #19720: [SPARK-22494][SQL] Fix 64KB limit exception with Coalesc...

2017-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19720 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83707/ Test FAILed. ---

[GitHub] spark issue #19720: [SPARK-22494][SQL] Fix 64KB limit exception with Coalesc...

2017-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19720 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19719: [SPARK-22487][SQL][followup] still keep spark.sql.hive.v...

2017-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19719 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19719: [SPARK-22487][SQL][followup] still keep spark.sql.hive.v...

2017-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19719 **[Test build #83708 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83708/testReport)** for PR 19719 at commit [`0ce241d`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #19719: [SPARK-22487][SQL][followup] still keep spark.sql.hive.v...

2017-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19719 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83708/ Test FAILed. ---

[GitHub] spark issue #19719: [SPARK-22487][SQL][followup] still keep spark.sql.hive.v...

2017-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19719 **[Test build #83705 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83705/testReport)** for PR 19719 at commit [`7b767bf`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #19719: [SPARK-22487][SQL][followup] still keep spark.sql.hive.v...

2017-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19719 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19719: [SPARK-22487][SQL][followup] still keep spark.sql.hive.v...

2017-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19719 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83705/ Test FAILed. ---

[GitHub] spark issue #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark DataFra...

2017-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19459 **[Test build #83703 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83703/testReport)** for PR 19459 at commit [`6c72e37`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark DataFra...

2017-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19459 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark DataFra...

2017-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19459 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83703/ Test PASSed. ---

[GitHub] spark issue #19407: [SPARK-21667][Streaming] ConsoleSink should not fail str...

2017-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19407 **[Test build #83704 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83704/testReport)** for PR 19407 at commit [`788fbf3`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #19407: [SPARK-21667][Streaming] ConsoleSink should not fail str...

2017-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19407 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83704/ Test PASSed. ---

[GitHub] spark issue #19407: [SPARK-21667][Streaming] ConsoleSink should not fail str...

2017-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19407 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19407: [SPARK-21667][Streaming] ConsoleSink should not fail str...

2017-11-10 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/19407 Thanks! Merging to master and 2.2. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-

[GitHub] spark pull request #19719: [SPARK-22487][SQL][followup] still keep spark.sql...

2017-11-10 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19719#discussion_r150358375 --- Diff: sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suites.scala --- @@ -521,7 +521,20 @@ class HiveT

[GitHub] spark pull request #19407: [SPARK-21667][Streaming] ConsoleSink should not f...

2017-11-10 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19407 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19077: [SPARK-21860][core]Improve memory reuse for heap memory ...

2017-11-10 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19077 According to [this material](http://s3-eu-west-1.amazonaws.com/presentations2013/10_presentation.pdf#pages=42), `Unsafe.allocateMemory` uses 8-byte boundary. --- ---

[GitHub] spark issue #19601: [SPARK-22383][SQL] Generate code to directly get value o...

2017-11-10 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19601 @cloud-fan could you please review this again? Now, this PR does not apply any change to `ColumnVector` and `WritableColumnVector`. --- --

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-10 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150361985 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -0,0 +1,236 @@ +/* + * Licensed to the Apache Software Foundatio

[GitHub] spark pull request #19719: [SPARK-22487][SQL][followup] still keep spark.sql...

2017-11-10 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19719#discussion_r150362312 --- Diff: sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suites.scala --- @@ -521,7 +521,20 @@ class HiveTh

[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...

2017-11-10 Thread sergiobilello-eb
Github user sergiobilello-eb commented on the issue: https://github.com/apache/spark/pull/18134 This function is not reported in the API documentation: https://spark.apache.org/docs/2.2.0/api/java/index.html?org/apache/spark/sql/functions.html or https://spark.apache.org/docs/2.1.0/a

[GitHub] spark issue #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new ORCFileF...

2017-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19651 **[Test build #83710 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83710/testReport)** for PR 19651 at commit [`4b09faa`](https://github.com/apache/spark/commit/4b

[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...

2017-11-10 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18134 @sergiobilello-eb This is just a SQL function. You can call it in SQL interface or using the `df.select(expr("dayofweek('2009-07-30')"))`. It is not part of the DataFrame functions. You can submi

[GitHub] spark pull request #19719: [SPARK-22487][SQL][followup] still keep spark.sql...

2017-11-10 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19719#discussion_r150363309 --- Diff: sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suites.scala --- @@ -521,7 +521,20 @@ class HiveT

[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...

2017-11-10 Thread sergiobilello-eb
Github user sergiobilello-eb commented on the issue: https://github.com/apache/spark/pull/18134 thanks @gatorsmile :) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mai

[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...

2017-11-10 Thread sergiobilello-eb
Github user sergiobilello-eb commented on the issue: https://github.com/apache/spark/pull/18134 ```spark-sql> select dayofweek('2007-02-03'), dayofweek('2009-07-30'), dayofweek('2017-05-27'), dayofweek(null), dayofweek('1582-10-15 13:10:15'); 17/11/10 16:08:23 INFO SparkSqlParser:

[GitHub] spark issue #19689: [SPARK-22462][SQL] Make rdd-based actions in Dataset tra...

2017-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19689 **[Test build #83711 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83711/testReport)** for PR 19689 at commit [`20dc217`](https://github.com/apache/spark/commit/20

[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...

2017-11-10 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18134 https://issues.apache.org/jira/browse/SPARK-20909 This is not part of 2.2. Based on JIRA, it will be included in 2.3 --- --

[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...

2017-11-10 Thread sergiobilello-eb
Github user sergiobilello-eb commented on the issue: https://github.com/apache/spark/pull/18134 thanks @gatorsmile! Do you suggest any workaround until then? I mean not rebuilding spark with that patch Can I register my UDF that contains that logic? ---

[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...

2017-11-10 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18134 Yeah, you always can implement such a UDF. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19719: [SPARK-22487][SQL][followup] still keep spark.sql...

2017-11-10 Thread yaooqinn
Github user yaooqinn commented on a diff in the pull request: https://github.com/apache/spark/pull/19719#discussion_r150364544 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala --- @@ -66,6 +66,12 @@ private[spark] object HiveUtils extends Logging {

[GitHub] spark pull request #19719: [SPARK-22487][SQL][followup] still keep spark.sql...

2017-11-10 Thread yaooqinn
Github user yaooqinn commented on a diff in the pull request: https://github.com/apache/spark/pull/19719#discussion_r150364666 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala --- @@ -66,6 +66,12 @@ private[spark] object HiveUtils extends Logging {

[GitHub] spark issue #19719: [SPARK-22487][SQL][followup] still keep spark.sql.hive.v...

2017-11-10 Thread yaooqinn
Github user yaooqinn commented on the issue: https://github.com/apache/spark/pull/19719 make it an AlternativeConfig maybe better --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comm

[GitHub] spark issue #19640: [SPARK-16986][CORE][WEB-UI] Support configure history se...

2017-11-10 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/19640 @cloud-fan For the UI part, how about this PR: https://github.com/apache/spark/pull/14577 --- - To unsubscribe, e-mail: reviews-

[GitHub] spark pull request #19719: [SPARK-22487][SQL][followup] still keep spark.sql...

2017-11-10 Thread yaooqinn
Github user yaooqinn commented on a diff in the pull request: https://github.com/apache/spark/pull/19719#discussion_r150366200 --- Diff: sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suites.scala --- @@ -521,7 +521,20 @@ class HiveThr

[GitHub] spark issue #19719: [SPARK-22487][SQL][followup] still keep spark.sql.hive.v...

2017-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19719 **[Test build #83712 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83712/testReport)** for PR 19719 at commit [`28fab02`](https://github.com/apache/spark/commit/28

[GitHub] spark pull request #19719: [SPARK-22487][SQL][followup] still keep spark.sql...

2017-11-10 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19719#discussion_r150366512 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala --- @@ -66,6 +66,12 @@ private[spark] object HiveUtils extends Logging {

[GitHub] spark pull request #19719: [SPARK-22487][SQL][followup] still keep spark.sql...

2017-11-10 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19719#discussion_r150366571 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala --- @@ -66,6 +66,12 @@ private[spark] object HiveUtils extends Logging {

[GitHub] spark issue #19640: [SPARK-16986][CORE][WEB-UI] Support configure history se...

2017-11-10 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19640 I think it makes sense, but the implementation is too hacky. We should extract the time epoch and convert it to timestamp string with local timezone. ---

[GitHub] spark pull request #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new O...

2017-11-10 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19651#discussion_r150367100 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcDeserializer.scala --- @@ -0,0 +1,206 @@ +/* + * Licensed to

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-10 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150367190 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/HadoopUtils.scala --- @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Software Foundatio

[GitHub] spark pull request #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new O...

2017-11-10 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19651#discussion_r150367382 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcDeserializer.scala --- @@ -0,0 +1,206 @@ +/* + * Licensed to

[GitHub] spark pull request #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new O...

2017-11-10 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19651#discussion_r150367580 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFilters.scala --- @@ -0,0 +1,183 @@ +/* + * Licensed to the A

[GitHub] spark pull request #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new O...

2017-11-10 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19651#discussion_r150367867 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcSerializer.scala --- @@ -0,0 +1,163 @@ +/* + * Licensed to th

[GitHub] spark pull request #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new O...

2017-11-10 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19651#discussion_r150367926 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcUtils.scala --- @@ -0,0 +1,131 @@ +/* + * Licensed to the Apa

[GitHub] spark issue #19719: [SPARK-22487][SQL][followup] still keep spark.sql.hive.v...

2017-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19719 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83712/ Test FAILed. ---

[GitHub] spark issue #19719: [SPARK-22487][SQL][followup] still keep spark.sql.hive.v...

2017-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19719 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19719: [SPARK-22487][SQL][followup] still keep spark.sql.hive.v...

2017-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19719 **[Test build #83712 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83712/testReport)** for PR 19719 at commit [`28fab02`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-10 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/19439 Thanks for the updates! My only remaining comments are about: * Default arguments for readImages in Scala not being Java-friendly (I'd still recommend taking the easy route by having 1 simple

[GitHub] spark pull request #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new O...

2017-11-10 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19651#discussion_r150368108 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcSerializer.scala --- @@ -0,0 +1,163 @@ +/* + * Licensed to th

[GitHub] spark pull request #19719: [SPARK-22487][SQL][followup] still keep spark.sql...

2017-11-10 Thread yaooqinn
Github user yaooqinn commented on a diff in the pull request: https://github.com/apache/spark/pull/19719#discussion_r150368111 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala --- @@ -66,6 +66,12 @@ private[spark] object HiveUtils extends Logging {

[GitHub] spark pull request #19714: [SPARK-22489][SQL] Shouldn't change broadcast joi...

2017-11-10 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/19714#discussion_r150368276 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -154,12 +158,12 @@ abstract class SparkStrategies extends Que

[GitHub] spark issue #19719: [SPARK-22487][SQL][followup] still keep spark.sql.hive.v...

2017-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19719 **[Test build #83713 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83713/testReport)** for PR 19719 at commit [`42d05a7`](https://github.com/apache/spark/commit/42

[GitHub] spark issue #19630: wip: [SPARK-22409] Introduce function type argument in p...

2017-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19630 **[Test build #83709 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83709/testReport)** for PR 19630 at commit [`94cded6`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #19630: wip: [SPARK-22409] Introduce function type argument in p...

2017-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19630 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83709/ Test FAILed. ---

[GitHub] spark issue #19630: wip: [SPARK-22409] Introduce function type argument in p...

2017-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19630 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #19719: [SPARK-22487][SQL][followup] still keep spark.sql...

2017-11-10 Thread yaooqinn
Github user yaooqinn commented on a diff in the pull request: https://github.com/apache/spark/pull/19719#discussion_r150368620 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLEnv.scala --- @@ -55,6 +55,7 @@ private[hive] object SparkSQ

[GitHub] spark issue #19714: [SPARK-22489][SQL] Shouldn't change broadcast join build...

2017-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19714 **[Test build #83714 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83714/testReport)** for PR 19714 at commit [`c78951b`](https://github.com/apache/spark/commit/c7

[GitHub] spark pull request #19719: [SPARK-22487][SQL][followup] still keep spark.sql...

2017-11-10 Thread yaooqinn
Github user yaooqinn commented on a diff in the pull request: https://github.com/apache/spark/pull/19719#discussion_r150368744 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala --- @@ -66,6 +66,12 @@ private[spark] object HiveUtils extends Logging {

[GitHub] spark issue #19719: [SPARK-22487][SQL][followup] still keep spark.sql.hive.v...

2017-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19719 **[Test build #83713 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83713/testReport)** for PR 19719 at commit [`42d05a7`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #19719: [SPARK-22487][SQL][followup] still keep spark.sql.hive.v...

2017-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19719 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83713/ Test FAILed. ---

[GitHub] spark pull request #19719: [SPARK-22487][SQL][followup] still keep spark.sql...

2017-11-10 Thread yaooqinn
Github user yaooqinn commented on a diff in the pull request: https://github.com/apache/spark/pull/19719#discussion_r150368818 --- Diff: sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suites.scala --- @@ -521,7 +521,20 @@ class HiveThr

[GitHub] spark issue #19719: [SPARK-22487][SQL][followup] still keep spark.sql.hive.v...

2017-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19719 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new O...

2017-11-10 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19651#discussion_r150369899 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFilters.scala --- @@ -0,0 +1,183 @@ +/* + * Licensed to t

[GitHub] spark issue #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new ORCFileF...

2017-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19651 **[Test build #83710 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83710/testReport)** for PR 19651 at commit [`4b09faa`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new ORCFileF...

2017-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19651 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19651: [SPARK-20682][SPARK-15474][SPARK-21791] Add new ORCFileF...

2017-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19651 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83710/ Test FAILed. ---

[GitHub] spark issue #19689: [SPARK-22462][SQL] Make rdd-based actions in Dataset tra...

2017-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19689 **[Test build #83711 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83711/testReport)** for PR 19689 at commit [`20dc217`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #19689: [SPARK-22462][SQL] Make rdd-based actions in Dataset tra...

2017-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19689 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19689: [SPARK-22462][SQL] Make rdd-based actions in Dataset tra...

2017-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19689 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83711/ Test PASSed. ---

[GitHub] spark pull request #19208: [SPARK-21087] [ML] CrossValidator, TrainValidatio...

2017-11-10 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/19208#discussion_r150373516 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tuning/CrossValidator.scala --- @@ -117,6 +123,12 @@ class CrossValidator @Since("1.2.0") (@Since("1.4.

[GitHub] spark issue #19714: [SPARK-22489][SQL] Shouldn't change broadcast join build...

2017-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19714 **[Test build #83714 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83714/testReport)** for PR 19714 at commit [`c78951b`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #19714: [SPARK-22489][SQL] Shouldn't change broadcast join build...

2017-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19714 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19714: [SPARK-22489][SQL] Shouldn't change broadcast join build...

2017-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19714 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83714/ Test PASSed. ---

[GitHub] spark issue #19077: [SPARK-21860][core]Improve memory reuse for heap memory ...

2017-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19077 **[Test build #83715 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83715/testReport)** for PR 19077 at commit [`967c73a`](https://github.com/apache/spark/commit/96

[GitHub] spark issue #19690: [SPARK-22467]Added a switch to support whether `stdout_s...

2017-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19690 **[Test build #83716 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83716/testReport)** for PR 19690 at commit [`37fba60`](https://github.com/apache/spark/commit/37

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150375592 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -0,0 +1,239 @@ +/* + * Licensed to the Apache Software Foundat

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150375543 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -0,0 +1,239 @@ +/* + * Licensed to the Apache Software Foundat

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150375535 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -0,0 +1,239 @@ +/* + * Licensed to the Apache Software Foundat

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150375473 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,196 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contri

[GitHub] spark issue #19713: [SPARK-22488] [SQL] Fix the view resolution issue in the...

2017-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19713 **[Test build #83717 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83717/testReport)** for PR 19713 at commit [`c491974`](https://github.com/apache/spark/commit/c4

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150376396 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,196 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contri

[GitHub] spark pull request #19543: [SPARK-19606][MESOS] Support constraints in spark...

2017-11-10 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/19543#discussion_r150376571 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/deploy/mesos/config.scala --- @@ -122,4 +122,11 @@ package object config {

<    1   2   3   4   5   >