[GitHub] spark pull request #19083: [SPARK-21871][SQL] Check actual bytecode size whe...

2017-10-01 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19083#discussion_r142059729 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala --- @@ -432,25 +432,31 @@ class DataFrameAggregateSuite extends

[GitHub] spark pull request #19083: [SPARK-21871][SQL] Check actual bytecode size whe...

2017-10-01 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19083#discussion_r142059351 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala --- @@ -432,25 +432,31 @@ class DataFrameAggregateSuite extends

[GitHub] spark pull request #19405: [SPARK-22178] [SQL] Refresh Persistent Views by R...

2017-10-01 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19405#discussion_r142059110 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/CatalogImpl.scala --- @@ -474,13 +474,20 @@ class CatalogImpl(sparkSession: SparkSession)

[GitHub] spark pull request #19083: [SPARK-21871][SQL] Check actual bytecode size whe...

2017-10-01 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19083#discussion_r142058947 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -1092,13 +1082,30 @@ object

[GitHub] spark pull request #19083: [SPARK-21871][SQL] Check actual bytecode size whe...

2017-10-01 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19083#discussion_r142058941 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala --- @@ -432,25 +432,31 @@ class DataFrameAggregateSuite extends

[GitHub] spark pull request #19083: [SPARK-21871][SQL] Check actual bytecode size whe...

2017-10-01 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19083#discussion_r142058817 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -1092,13 +1082,30 @@ object

[GitHub] spark pull request #19083: [SPARK-21871][SQL] Check actual bytecode size whe...

2017-10-01 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19083#discussion_r142058550 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -1020,6 +1006,10 @@ abstract class

[GitHub] spark pull request #19083: [SPARK-21871][SQL] Check actual bytecode size whe...

2017-10-01 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19083#discussion_r142058540 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -1092,13 +1082,30 @@ object

[GitHub] spark pull request #19083: [SPARK-21871][SQL] Check actual bytecode size whe...

2017-10-01 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19083#discussion_r142058489 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -1020,6 +1006,10 @@ abstract class

[GitHub] spark pull request #19083: [SPARK-21871][SQL] Check actual bytecode size whe...

2017-10-01 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19083#discussion_r142058408 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -1092,13 +1082,30 @@ object

[GitHub] spark pull request #19083: [SPARK-21871][SQL] Check actual bytecode size whe...

2017-10-01 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19083#discussion_r142058372 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -1092,13 +1082,30 @@ object

[GitHub] spark pull request #19083: [SPARK-21871][SQL] Check actual bytecode size whe...

2017-10-01 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19083#discussion_r142058236 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -1092,13 +1082,30 @@ object

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-01 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142057939 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowEvalPythonExec.scala --- @@ -44,14 +66,24 @@ case class

[GitHub] spark issue #17702: [SPARK-20408][SQL] Get the glob path in parallel to redu...

2017-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17702 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #17702: [SPARK-20408][SQL] Get the glob path in parallel to redu...

2017-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17702 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82378/ Test PASSed. ---

[GitHub] spark issue #17702: [SPARK-20408][SQL] Get the glob path in parallel to redu...

2017-10-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17702 **[Test build #82378 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82378/testReport)** for PR 17702 at commit

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-01 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142055482 --- Diff: python/pyspark/sql/functions.py --- @@ -2181,31 +2186,69 @@ def udf(f=None, returnType=StringType()): @since(2.3) def pandas_udf(f=None,

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-01 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142055226 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/FlatMapGroupsInPandasExec.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed to

[GitHub] spark pull request #19392: [SPARK-22169][SQL] table name with numbers and ch...

2017-10-01 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19392#discussion_r142054949 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -510,11 +510,15 @@ rowFormat ;

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2017-10-01 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18931 ping @cloud-fan @gatorsmile Please take a look for review. Thanks. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19083: [SPARK-21871][SQL] Check actual bytecode size whe...

2017-10-01 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19083#discussion_r142053683 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -1092,13 +1082,30 @@ object

[GitHub] spark pull request #19083: [SPARK-21871][SQL] Check actual bytecode size whe...

2017-10-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19083#discussion_r142053181 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -1092,13 +1082,30 @@ object

[GitHub] spark issue #18732: [SPARK-20396][SQL][PySpark] groupby().apply() with panda...

2017-10-01 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/18732 @HyukjinKwon Thanks for the feedback. I will address those and update tomorrow. --- - To unsubscribe, e-mail:

[GitHub] spark issue #18732: [SPARK-20396][SQL][PySpark] groupby().apply() with panda...

2017-10-01 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/18732 @rxin, `transform` takes a function: pd.Series -> pd.Series and apply the function on all columns: ``` df.show() id v1 v2 v3 a 1.0 4.0 0.0 a 2.0 5.0

[GitHub] spark issue #19406: [SPARK-22179] percentile_approx should choose the first ...

2017-10-01 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/19406 @srowen For example, given input data 1 to 10, if a user queries 10% (or even less) percentile, it should return 1, because the first value 1 already reaches 10% percentage. Without this fix, it

[GitHub] spark issue #19083: [SPARK-21871][SQL] Check actual bytecode size when compi...

2017-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19083 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82375/ Test PASSed. ---

[GitHub] spark issue #19083: [SPARK-21871][SQL] Check actual bytecode size when compi...

2017-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19083 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19083: [SPARK-21871][SQL] Check actual bytecode size when compi...

2017-10-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19083 **[Test build #82375 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82375/testReport)** for PR 19083 at commit

[GitHub] spark issue #17862: [SPARK-20602] [ML]Adding LBFGS optimizer and Squared_hin...

2017-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17862 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #17862: [SPARK-20602] [ML]Adding LBFGS optimizer and Squared_hin...

2017-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17862 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82377/ Test PASSed. ---

[GitHub] spark issue #17862: [SPARK-20602] [ML]Adding LBFGS optimizer and Squared_hin...

2017-10-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17862 **[Test build #82377 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82377/testReport)** for PR 17862 at commit

[GitHub] spark issue #17702: [SPARK-20408][SQL] Get the glob path in parallel to redu...

2017-10-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17702 **[Test build #82378 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82378/testReport)** for PR 17702 at commit

[GitHub] spark issue #17702: [SPARK-20408][SQL] Get the glob path in parallel to redu...

2017-10-01 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/17702 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19408: [SPARK-22180][CORE] Allow IPv6 address in org.apa...

2017-10-01 Thread obermeier
Github user obermeier commented on a diff in the pull request: https://github.com/apache/spark/pull/19408#discussion_r142047496 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -981,7 +981,13 @@ private[spark] object Utils extends Logging { return

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2017-10-01 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19222 @hvanhovell could you please review this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18704: [SPARK-20783][SQL] Create ColumnVector to abstract exist...

2017-10-01 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18704 @cloud-fan could you please review this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #17862: [SPARK-20602] [ML]Adding LBFGS optimizer and Squared_hin...

2017-10-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17862 **[Test build #82377 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82377/testReport)** for PR 17862 at commit

[GitHub] spark issue #17862: [SPARK-20602] [ML]Adding LBFGS optimizer and Squared_hin...

2017-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17862 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #17862: [SPARK-20602] [ML]Adding LBFGS optimizer and Squared_hin...

2017-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17862 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82376/ Test FAILed. ---

[GitHub] spark issue #17862: [SPARK-20602] [ML]Adding LBFGS optimizer and Squared_hin...

2017-10-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17862 **[Test build #82376 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82376/testReport)** for PR 17862 at commit

[GitHub] spark issue #17862: [SPARK-20602] [ML]Adding LBFGS optimizer and Squared_hin...

2017-10-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17862 **[Test build #82376 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82376/testReport)** for PR 17862 at commit

[GitHub] spark issue #19083: [SPARK-21871][SQL] Check actual bytecode size when compi...

2017-10-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19083 **[Test build #82375 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82375/testReport)** for PR 19083 at commit

[GitHub] spark issue #17359: [SPARK-20028][SQL] Add aggreagate expression nGrams

2017-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17359 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19403: [R][BUILD][WIP] test

2017-10-01 Thread felixcheung
Github user felixcheung closed the pull request at: https://github.com/apache/spark/pull/19403 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19408: [SPARK-22180][CORE] Allow IPv6 address in org.apa...

2017-10-01 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19408#discussion_r142038575 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -981,7 +981,13 @@ private[spark] object Utils extends Logging { return

[GitHub] spark pull request #19408: [SPARK-22180][CORE] Allow IPv6 address in org.apa...

2017-10-01 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19408#discussion_r142038567 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -981,7 +981,13 @@ private[spark] object Utils extends Logging { return

[GitHub] spark pull request #19408: [SPARK-22180][CORE] Allow IPv6 address in org.apa...

2017-10-01 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19408#discussion_r142038556 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -981,7 +981,13 @@ private[spark] object Utils extends Logging { return

[GitHub] spark pull request #19408: [SPARK-22180][CORE] Allow IPv6 address in org.apa...

2017-10-01 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19408#discussion_r142038519 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -981,7 +981,13 @@ private[spark] object Utils extends Logging { return

[GitHub] spark pull request #19083: [SPARK-21871][SQL] Check actual bytecode size whe...

2017-10-01 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19083#discussion_r142037322 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/AggregateBenchmark.scala --- @@ -301,10 +301,10 @@ class AggregateBenchmark

[GitHub] spark issue #19083: [SPARK-21871][SQL] Check actual bytecode size when compi...

2017-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19083 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19083: [SPARK-21871][SQL] Check actual bytecode size when compi...

2017-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19083 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82373/ Test FAILed. ---

[GitHub] spark issue #19083: [SPARK-21871][SQL] Check actual bytecode size when compi...

2017-10-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19083 **[Test build #82373 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82373/testReport)** for PR 19083 at commit

[GitHub] spark pull request #19229: [SPARK-22001][ML][SQL] ImputerModel can do withCo...

2017-10-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19229 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19405: [SPARK-22178] [SQL] Refresh Persistent Views by REFRESH ...

2017-10-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19405 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19229: [SPARK-22001][ML][SQL] ImputerModel can do withColumn fo...

2017-10-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19229 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19401: [SPARK-22176][SQL] Fix overflow issue in Dataset....

2017-10-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19401#discussion_r142035876 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -237,7 +237,7 @@ class Dataset[T] private[sql]( */

[GitHub] spark issue #19083: [SPARK-21871][SQL] Check actual bytecode size when compi...

2017-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19083 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19083: [SPARK-21871][SQL] Check actual bytecode size when compi...

2017-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19083 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82374/ Test FAILed. ---

[GitHub] spark issue #19083: [SPARK-21871][SQL] Check actual bytecode size when compi...

2017-10-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19083 **[Test build #82374 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82374/testReport)** for PR 19083 at commit

[GitHub] spark issue #19083: [SPARK-21871][SQL] Check actual bytecode size when compi...

2017-10-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19083 **[Test build #82374 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82374/testReport)** for PR 19083 at commit

[GitHub] spark issue #19083: [SPARK-21871][SQL] Check actual bytecode size when compi...

2017-10-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19083 **[Test build #82373 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82373/testReport)** for PR 19083 at commit

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19181 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82372/ Test PASSed. ---

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19181 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-10-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19181 **[Test build #82372 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82372/testReport)** for PR 19181 at commit

[GitHub] spark issue #19408: [SPARK-22180][CORE] Allow IPv6

2017-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19408 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19408: [SPARK-22180][CORE] Allow IPv6

2017-10-01 Thread obermeier
GitHub user obermeier opened a pull request: https://github.com/apache/spark/pull/19408 [SPARK-22180][CORE] Allow IPv6 External applications like Apache Cassandra are able to deal with IPv6 addresses. Libraries like spark-cassandra-connector combine Apache Cassandra with Apache

[GitHub] spark issue #19399: [SPARK-22175][WEB-UI] Add status column to history page

2017-10-01 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19399 @jerryshao @squito Could you help review this?Thanks --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #19401: [SPARK-22176][SQL] Fix overflow issue in Dataset....

2017-10-01 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19401#discussion_r142031417 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -237,7 +237,7 @@ class Dataset[T] private[sql]( */ private[sql]

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142029595 --- Diff: python/pyspark/sql/functions.py --- @@ -2181,31 +2186,69 @@ def udf(f=None, returnType=StringType()): @since(2.3) def

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142029720 --- Diff: python/pyspark/sql/functions.py --- @@ -2181,31 +2186,69 @@ def udf(f=None, returnType=StringType()): @since(2.3) def

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142029736 --- Diff: python/pyspark/sql/group.py --- @@ -194,6 +194,37 @@ def pivot(self, pivot_col, values=None): jgd =

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142029696 --- Diff: python/pyspark/sql/functions.py --- @@ -2181,31 +2186,69 @@ def udf(f=None, returnType=StringType()): @since(2.3) def

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142029802 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/FlatMapGroupsInPandasExec.scala --- @@ -0,0 +1,91 @@ +/* + * Licensed

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142029714 --- Diff: python/pyspark/sql/functions.py --- @@ -2181,31 +2186,69 @@ def udf(f=None, returnType=StringType()): @since(2.3) def

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142029786 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/FlatMapGroupsInPandasExec.scala --- @@ -0,0 +1,91 @@ +/* + * Licensed

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142029655 --- Diff: python/pyspark/sql/functions.py --- @@ -2181,31 +2186,69 @@ def udf(f=None, returnType=StringType()): @since(2.3) def

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-10-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19181 **[Test build #82372 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82372/testReport)** for PR 19181 at commit

[GitHub] spark issue #19407: [SPARK-21667][Streaming] ConsoleSink should not fail str...

2017-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19407 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82371/ Test FAILed. ---

[GitHub] spark issue #19407: [SPARK-21667][Streaming] ConsoleSink should not fail str...

2017-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19407 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19407: [SPARK-21667][Streaming] ConsoleSink should not fail str...

2017-10-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19407 **[Test build #82371 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82371/testReport)** for PR 19407 at commit

[GitHub] spark issue #19404: [SPARK-21760] [Streaming] Fix for Structured streaming t...

2017-10-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19404 **[Test build #3940 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3940/testReport)** for PR 19404 at commit

[GitHub] spark issue #19290: [SPARK-22063][R] Fixes lint check failures in R by lates...

2017-10-01 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19290 So, @felixcheung, @shaneknapp and @shivaram, looks we have comments, https://github.com/apache/spark/pull/19290#issuecomment-21991 and

[GitHub] spark pull request #19290: [SPARK-22063][R] Fixes lint check failures in R b...

2017-10-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19290 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19290: [SPARK-22063][R] Fixes lint check failures in R by lates...

2017-10-01 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19290 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19290: [SPARK-22063][R] Fixes lint check failures in R by lates...

2017-10-01 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19290 Let me merge this one first. This shouldn't cause any problem to built system for now. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19402: [SPARK-22167][R][BUILD] sparkr packaging issue al...

2017-10-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19402#discussion_r142024980 --- Diff: core/pom.xml --- @@ -499,7 +499,7 @@ -

[GitHub] spark issue #19229: [SPARK-22001][ML][SQL] ImputerModel can do withColumn fo...

2017-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19229 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82369/ Test PASSed. ---

[GitHub] spark issue #19229: [SPARK-22001][ML][SQL] ImputerModel can do withColumn fo...

2017-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19229 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19406: [SPARK-22179] percentile_approx should choose the first ...

2017-10-01 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19406 The JIRA doesn't explain what this is meant to fix. What case does this help get more correct? --- - To unsubscribe, e-mail:

[GitHub] spark issue #19229: [SPARK-22001][ML][SQL] ImputerModel can do withColumn fo...

2017-10-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19229 **[Test build #82369 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82369/testReport)** for PR 19229 at commit

[GitHub] spark issue #19404: [SPARK-21760] [Streaming] Fix for Structured streaming t...

2017-10-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19404 **[Test build #3940 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3940/testReport)** for PR 19404 at commit

[GitHub] spark pull request #19401: [SPARK-22176][SQL] Fix overflow issue in Dataset....

2017-10-01 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19401#discussion_r142023186 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -237,7 +237,7 @@ class Dataset[T] private[sql]( */ private[sql]

[GitHub] spark pull request #19407: [SPARK-21667][Streaming] ConsoleSink should not f...

2017-10-01 Thread rekhajoshm
Github user rekhajoshm commented on a diff in the pull request: https://github.com/apache/spark/pull/19407#discussion_r142023140 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala --- @@ -269,7 +269,7 @@ final class DataStreamWriter[T]

[GitHub] spark issue #19407: [SPARK-21667][Streaming] ConsoleSink should not fail str...

2017-10-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19407 **[Test build #82371 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82371/testReport)** for PR 19407 at commit

[GitHub] spark issue #19407: [SPARK-21667][Streaming] ConsoleSink should not fail str...

2017-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19407 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82370/ Test FAILed. ---

[GitHub] spark issue #19407: [SPARK-21667][Streaming] ConsoleSink should not fail str...

2017-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19407 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19407: [SPARK-21667][Streaming] ConsoleSink should not fail str...

2017-10-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19407 **[Test build #82370 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82370/testReport)** for PR 19407 at commit

[GitHub] spark pull request #19407: [SPARK-21667][Streaming] ConsoleSink should not f...

2017-10-01 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/19407#discussion_r142022819 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala --- @@ -269,7 +269,7 @@ final class DataStreamWriter[T]

[GitHub] spark issue #19407: [SPARK-21667][Streaming] ConsoleSink should not fail str...

2017-10-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19407 **[Test build #82370 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82370/testReport)** for PR 19407 at commit

[GitHub] spark pull request #19407: [SPARK-21667][Streaming] ConsoleSink should not f...

2017-10-01 Thread rekhajoshm
GitHub user rekhajoshm opened a pull request: https://github.com/apache/spark/pull/19407 [SPARK-21667][Streaming] ConsoleSink should not fail streaming query with checkpointLocation option ## What changes were proposed in this pull request? Fix to allow recovery on console ,

  1   2   >