[GitHub] spark issue #18732: [SPARK-20396][SQL][PySpark] groupby().apply() with panda...

2017-10-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18732 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18732: [SPARK-20396][SQL][PySpark] groupby().apply() with panda...

2017-10-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18732 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82457/ Test FAILed. ---

[GitHub] spark issue #18732: [SPARK-20396][SQL][PySpark] groupby().apply() with panda...

2017-10-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18732 **[Test build #82457 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82457/testReport)** for PR 18732 at commit

[GitHub] spark issue #18748: [SPARK-20679][ML] Support recommending for a subset of u...

2017-10-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18748 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82458/ Test PASSed. ---

[GitHub] spark issue #18748: [SPARK-20679][ML] Support recommending for a subset of u...

2017-10-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18748 **[Test build #82458 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82458/testReport)** for PR 18748 at commit

[GitHub] spark issue #18748: [SPARK-20679][ML] Support recommending for a subset of u...

2017-10-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18748 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19412: [SPARK-22142][BUILD][STREAMING] Move Flume support behin...

2017-10-04 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19412 I'm not really convinced that adding this profile is really achieving much. After all, the flume connector doesn't end up in the Spark packaging, only in maven central, and the new profile seems to

[GitHub] spark issue #19416: [SPARK-22187][SS] Update unsaferow format for saved stat...

2017-10-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19416 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82454/ Test PASSed. ---

[GitHub] spark issue #19416: [SPARK-22187][SS] Update unsaferow format for saved stat...

2017-10-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19416 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19416: [SPARK-22187][SS] Update unsaferow format for saved stat...

2017-10-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19416 **[Test build #82454 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82454/testReport)** for PR 19416 at commit

[GitHub] spark issue #19108: [SPARK-21898][ML] Feature parity for KolmogorovSmirnovTe...

2017-10-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19108 **[Test build #82459 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82459/testReport)** for PR 19108 at commit

[GitHub] spark pull request #19392: [SPARK-22169][SQL] support byte length literal as...

2017-10-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19392 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19392: [SPARK-22169][SQL] support byte length literal as identi...

2017-10-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19392 Thanks! Merged to master. Could you backport it to 2.2? --- - To unsubscribe, e-mail:

[GitHub] spark issue #16976: [SPARK-19610][SQL] Support parsing multiline CSV files

2017-10-04 Thread smram
Github user smram commented on the issue: https://github.com/apache/spark/pull/16976 > don't think it quite works +1 i could not get this to work either @raviolli Does it need the univocity parser to be specified as suggested initially here

[GitHub] spark issue #18748: [SPARK-20679][ML] Support recommending for a subset of u...

2017-10-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18748 **[Test build #82458 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82458/testReport)** for PR 18748 at commit

[GitHub] spark issue #18748: [SPARK-20679][ML] Support recommending for a subset of u...

2017-10-04 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/18748 Jenkins retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #18748: [SPARK-20679][ML] Support recommending for a subset of u...

2017-10-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18748 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82456/ Test FAILed. ---

[GitHub] spark issue #18748: [SPARK-20679][ML] Support recommending for a subset of u...

2017-10-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18748 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18748: [SPARK-20679][ML] Support recommending for a subset of u...

2017-10-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18748 **[Test build #82456 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82456/testReport)** for PR 18748 at commit

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-04 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142770337 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/FlatMapGroupsInPandasExec.scala --- @@ -0,0 +1,89 @@ +/* + * Licensed

[GitHub] spark issue #18747: [SPARK-20822][SQL] Generate code to directly get value f...

2017-10-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18747 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82453/ Test PASSed. ---

[GitHub] spark issue #18747: [SPARK-20822][SQL] Generate code to directly get value f...

2017-10-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18747 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18747: [SPARK-20822][SQL] Generate code to directly get value f...

2017-10-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18747 **[Test build #82453 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82453/testReport)** for PR 18747 at commit

[GitHub] spark issue #19392: [SPARK-22169][SQL] support byte length literal as identi...

2017-10-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19392 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19392: [SPARK-22169][SQL] support byte length literal as identi...

2017-10-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19392 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82452/ Test PASSed. ---

[GitHub] spark issue #19392: [SPARK-22169][SQL] support byte length literal as identi...

2017-10-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19392 **[Test build #82452 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82452/testReport)** for PR 19392 at commit

[GitHub] spark issue #18748: [SPARK-20679][ML] Support recommending for a subset of u...

2017-10-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18748 **[Test build #82456 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82456/testReport)** for PR 18748 at commit

[GitHub] spark issue #18732: [SPARK-20396][SQL][PySpark] groupby().apply() with panda...

2017-10-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18732 **[Test build #82457 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82457/testReport)** for PR 18732 at commit

[GitHub] spark issue #18966: [SPARK-21751][SQL] CodeGeneraor.splitExpressions counts ...

2017-10-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18966 **[Test build #82455 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82455/testReport)** for PR 18966 at commit

[GitHub] spark pull request #18966: [SPARK-21751][SQL] CodeGeneraor.splitExpressions ...

2017-10-04 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18966#discussion_r142756140 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -769,16 +769,27 @@ class CodegenContext {

[GitHub] spark issue #19416: [SPARK-22187][SS] Update unsaferow format for saved stat...

2017-10-04 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/19416 LGTM pending tests --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19431: [SPARK-18580] [DStreams] [external/kafka-0-10][external/...

2017-10-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19431 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19430: Spark 18580

2017-10-04 Thread akonopko
Github user akonopko commented on the issue: https://github.com/apache/spark/pull/19430 @vanzin plz take a look at https://github.com/apache/spark/pull/19431 I am deleting this PR cause it was mistakenly created against branch-2.2 ---

[GitHub] spark issue #19430: Spark 18580

2017-10-04 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19430 Why isn't this PR against master? Is the change not needed there? The title also doesn't follow the Spark convention. ---

[GitHub] spark pull request #19430: Spark 18580

2017-10-04 Thread akonopko
Github user akonopko closed the pull request at: https://github.com/apache/spark/pull/19430 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19431: Add spark.streaming.backpressure.initialRate to d...

2017-10-04 Thread akonopko
GitHub user akonopko opened a pull request: https://github.com/apache/spark/pull/19431 Add spark.streaming.backpressure.initialRate to direct Kafka streams You can merge this pull request into a Git repository by running: $ git pull https://github.com/akonopko/spark

[GitHub] spark issue #19430: Spark 18580

2017-10-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19430 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19430: Spark 18580

2017-10-04 Thread akonopko
GitHub user akonopko reopened a pull request: https://github.com/apache/spark/pull/19430 Spark 18580 ## What changes were proposed in this pull request? Add `spark.streaming.backpressure.initialRate` to direct Kafka Streams for Kafka 0.8 and 0.10 This is required in

[GitHub] spark pull request #19430: Spark 18580

2017-10-04 Thread akonopko
Github user akonopko closed the pull request at: https://github.com/apache/spark/pull/19430 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19430: Spark 18580

2017-10-04 Thread akonopko
GitHub user akonopko opened a pull request: https://github.com/apache/spark/pull/19430 Spark 18580 ## What changes were proposed in this pull request? Add `spark.streaming.backpressure.initialRate` to direct Kafka Streams for Kafka 0.8 and 0.10 This is required in

[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data

2017-10-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19041 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82450/ Test FAILed. ---

[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data

2017-10-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19041 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data

2017-10-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19041 **[Test build #82450 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82450/testReport)** for PR 19041 at commit

[GitHub] spark issue #19416: [SPARK-22187][SS] Update unsaferow format for saved stat...

2017-10-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19416 **[Test build #82454 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82454/testReport)** for PR 19416 at commit

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-04 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142740947 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/RelationalGroupedDataset.scala --- @@ -435,6 +435,33 @@ class RelationalGroupedDataset

[GitHub] spark issue #19425: [SPARK-22196][Core] Combine multiple input splits into a...

2017-10-04 Thread vgankidi
Github user vgankidi commented on the issue: https://github.com/apache/spark/pull/19425 @davies Can you please take a look? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-04 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142735696 --- Diff: python/pyspark/sql/group.py --- @@ -194,6 +194,65 @@ def pivot(self, pivot_col, values=None): jgd =

[GitHub] spark pull request #19083: [SPARK-21871][SQL] Check actual bytecode size whe...

2017-10-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19083 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19083: [SPARK-21871][SQL] Check actual bytecode size when compi...

2017-10-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19083 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19392: [SPARK-22169][SQL] support byte length literal as identi...

2017-10-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19392 LGTM pending Jenkins --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19392: [SPARK-22169][SQL] support byte length literal as...

2017-10-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19392#discussion_r142731310 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -699,20 +699,30 @@ class AstBuilder(conf: SQLConf)

[GitHub] spark pull request #18033: [SPARK-20807][SQL] Add compression/decompression ...

2017-10-04 Thread kiszk
Github user kiszk closed the pull request at: https://github.com/apache/spark/pull/18033 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18747: [WIP][SPARK-20822][SQL] Generate code to directly get va...

2017-10-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18747 **[Test build #82453 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82453/testReport)** for PR 18747 at commit

[GitHub] spark issue #19392: [SPARK-22169][SQL] support byte length literal as identi...

2017-10-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19392 **[Test build #82452 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82452/testReport)** for PR 19392 at commit

[GitHub] spark issue #19272: [Spark-21842][Mesos] Support Kerberos ticket renewal and...

2017-10-04 Thread kalvinnchau
Github user kalvinnchau commented on the issue: https://github.com/apache/spark/pull/19272 @ArtRand curious, what version of hadoop are you build spark against and what version is the cluster that you're running? ---

[GitHub] spark issue #18747: [WIP][SPARK-20822][SQL] Generate code to directly get va...

2017-10-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18747 **[Test build #82451 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82451/testReport)** for PR 18747 at commit

[GitHub] spark issue #18747: [WIP][SPARK-20822][SQL] Generate code to directly get va...

2017-10-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18747 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18747: [WIP][SPARK-20822][SQL] Generate code to directly get va...

2017-10-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18747 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82451/ Test FAILed. ---

[GitHub] spark pull request #19251: [SPARK-22035][SQL]the value of statistical logica...

2017-10-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19251#discussion_r142722330 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/SizeInBytesOnlyStatsPlanVisitor.scala --- @@ -32,12

[GitHub] spark issue #18747: [WIP][SPARK-20822][SQL] Generate code to directly get va...

2017-10-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18747 **[Test build #82451 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82451/testReport)** for PR 18747 at commit

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-04 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142720877 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/FlatMapGroupsInPandasExec.scala --- @@ -0,0 +1,89 @@ +/* + * Licensed to

[GitHub] spark issue #18969: [SPARK-21520][SQL][FOLLOW-UP]fix a special case for non-...

2017-10-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18969 @heary-cao Maybe you can close this PR first? @jiangxb1987 will handle it in the previous PR. --- - To unsubscribe, e-mail:

[GitHub] spark issue #13893: [SPARK-14172][SQL] Hive table partition predicate not pa...

2017-10-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13893 @heary-cao tried to resolve the same issue in https://github.com/apache/spark/pull/18969 ping @jiangxb1987 --- -

[GitHub] spark pull request #19419: [SPARK-22188] [CORE] Adding security headers for ...

2017-10-04 Thread krishna-pandey
Github user krishna-pandey commented on a diff in the pull request: https://github.com/apache/spark/pull/19419#discussion_r142708896 --- Diff: conf/spark-defaults.conf.template --- @@ -19,9 +19,16 @@ # This is useful for setting default environmental settings. #

[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data

2017-10-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19041 **[Test build #82450 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82450/testReport)** for PR 19041 at commit

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-04 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142704642 --- Diff: python/pyspark/sql/group.py --- @@ -192,7 +193,66 @@ def pivot(self, pivot_col, values=None): jgd = self._jgd.pivot(pivot_col)

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-04 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142704126 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/RelationalGroupedDataset.scala --- @@ -435,6 +435,33 @@ class RelationalGroupedDataset

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-04 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142703829 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/RelationalGroupedDataset.scala --- @@ -435,6 +435,33 @@ class RelationalGroupedDataset

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-04 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142703487 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/RelationalGroupedDataset.scala --- @@ -435,6 +435,33 @@ class RelationalGroupedDataset

[GitHub] spark pull request #19419: [SPARK-22188] [CORE] Adding security headers for ...

2017-10-04 Thread krishna-pandey
Github user krishna-pandey commented on a diff in the pull request: https://github.com/apache/spark/pull/19419#discussion_r142701588 --- Diff: conf/spark-defaults.conf.template --- @@ -19,9 +19,16 @@ # This is useful for setting default environmental settings. #

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-04 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142697418 --- Diff: python/pyspark/worker.py --- @@ -74,17 +74,35 @@ def wrap_udf(f, return_type): def wrap_pandas_udf(f, return_type): -

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-04 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142695929 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/FlatMapGroupsInPandasExec.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-04 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142695843 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ExtractPythonUDFs.scala --- @@ -111,6 +111,9 @@ object ExtractPythonUDFs

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-04 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142695501 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowEvalPythonExec.scala --- @@ -26,6 +26,25 @@ import

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-04 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142695129 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/object.scala --- @@ -519,3 +519,18 @@ case class CoGroup(

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-04 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142694835 --- Diff: python/pyspark/worker.py --- @@ -74,17 +74,35 @@ def wrap_udf(f, return_type): def wrap_pandas_udf(f, return_type): -

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-04 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142694484 --- Diff: python/pyspark/worker.py --- @@ -74,17 +74,35 @@ def wrap_udf(f, return_type): def wrap_pandas_udf(f, return_type): -

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-04 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142694381 --- Diff: python/pyspark/worker.py --- @@ -74,17 +74,35 @@ def wrap_udf(f, return_type): def wrap_pandas_udf(f, return_type): -

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-04 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142693686 --- Diff: python/pyspark/worker.py --- @@ -74,17 +74,35 @@ def wrap_udf(f, return_type): def wrap_pandas_udf(f, return_type): -

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-04 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142693843 --- Diff: python/pyspark/worker.py --- @@ -74,17 +74,35 @@ def wrap_udf(f, return_type): def wrap_pandas_udf(f, return_type): -

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-04 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142692448 --- Diff: python/pyspark/sql/tests.py --- @@ -3106,8 +3106,9 @@ def assertFramesEqual(self, df_with_arrow, df_without):

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-04 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142691179 --- Diff: python/pyspark/sql/tests.py --- @@ -3106,8 +3106,9 @@ def assertFramesEqual(self, df_with_arrow, df_without):

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-04 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142690650 --- Diff: python/pyspark/sql/group.py --- @@ -192,7 +193,66 @@ def pivot(self, pivot_col, values=None): jgd = self._jgd.pivot(pivot_col)

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-04 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142690602 --- Diff: python/pyspark/sql/group.py --- @@ -192,7 +193,66 @@ def pivot(self, pivot_col, values=None): jgd = self._jgd.pivot(pivot_col)

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-04 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142689702 --- Diff: python/pyspark/sql/tests.py --- @@ -3106,8 +3106,9 @@ def assertFramesEqual(self, df_with_arrow, df_without):

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-10-04 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r142678914 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/FlatMapGroupsInPandasExec.scala --- @@ -0,0 +1,89 @@ +/* + * Licensed

[GitHub] spark issue #19429: [SPARK-20055] [Docs] Added documentation for loading csv...

2017-10-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19429 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19429: [SPARK-20055] [Docs] Added documentation for load...

2017-10-04 Thread jomach
GitHub user jomach opened a pull request: https://github.com/apache/spark/pull/19429 [SPARK-20055] [Docs] Added documentation for loading csv files into DataFrames ## What changes were proposed in this pull request? Added documentation for loading csv files

[GitHub] spark issue #19090: [SPARK-21877][DEPLOY, WINDOWS] Handle quotes in Windows ...

2017-10-04 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19090 Thanks for reviewing @jsnowacki, let me try to take a final look. I also checked what I could all but let me double check. Just want to be careful as it's the entry point. ---

[GitHub] spark issue #19090: [SPARK-21877][DEPLOY, WINDOWS] Handle quotes in Windows ...

2017-10-04 Thread jsnowacki
Github user jsnowacki commented on the issue: https://github.com/apache/spark/pull/19090 I think the comments are fine and sufficiently explain extra quotes existence. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19428: [SPARK-22131][MESOS] Mesos driver secrets

2017-10-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19428 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19428: [SPARK-22131][MESOS] Mesos driver secrets

2017-10-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19428 **[Test build #82449 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82449/testReport)** for PR 19428 at commit

[GitHub] spark issue #19428: [SPARK-22131][MESOS] Mesos driver secrets

2017-10-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19428 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82449/ Test FAILed. ---

[GitHub] spark issue #19428: [SPARK-22131][MESOS] Mesos driver secrets

2017-10-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19428 **[Test build #82449 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82449/testReport)** for PR 19428 at commit

[GitHub] spark issue #19090: [SPARK-21877][DEPLOY, WINDOWS] Handle quotes in Windows ...

2017-10-04 Thread minixalpha
Github user minixalpha commented on the issue: https://github.com/apache/spark/pull/19090 @jsnowacki I have already add comments to explain the quotes, could you help me review the comments? Thanks. --- - To

[GitHub] spark pull request #19428: [SPARK-22131][MESOS] Mesos driver secrets

2017-10-04 Thread susanxhuynh
GitHub user susanxhuynh opened a pull request: https://github.com/apache/spark/pull/19428 [SPARK-22131][MESOS] Mesos driver secrets ## What changes were proposed in this pull request? The driver launches executors that have access to env or file-based secrets. Most

[GitHub] spark issue #18924: [SPARK-14371] [MLLIB] OnlineLDAOptimizer should not coll...

2017-10-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18924 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82448/ Test PASSed. ---

[GitHub] spark issue #18924: [SPARK-14371] [MLLIB] OnlineLDAOptimizer should not coll...

2017-10-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18924 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18924: [SPARK-14371] [MLLIB] OnlineLDAOptimizer should not coll...

2017-10-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18924 **[Test build #82448 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82448/testReport)** for PR 18924 at commit

[GitHub] spark issue #19427: Reset spark.driver.bindAddress when starting a Checkpoin...

2017-10-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19427 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

<    1   2   3   >