[GitHub] spark issue #18005: [SPARK-20773][SQL] ParquetWriteSupport.writeFields is qu...

2017-05-16 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/18005 Can you also make sure that we do not use a `Seq` for struct writing? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #17467: [SPARK-20140][DStream] Remove hardcoded kinesis r...

2017-05-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17467 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18005: [SPARK-20773][SQL] ParquetWriteSupport.writeFields is qu...

2017-05-16 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/18005 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #17467: [SPARK-20140][DStream] Remove hardcoded kinesis retry wa...

2017-05-16 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/17467 LGTM! Merging to master/branch-2.2 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #17087: [SPARK-19372][SQL] Fix throwing a Java exception ...

2017-05-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17087 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17087: [SPARK-19372][SQL] Fix throwing a Java exception at df.f...

2017-05-16 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/17087 LGTM. Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #17999: [SPARK-20751][SQL] Add built-in SQL Function - CO...

2017-05-16 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17999#discussion_r116860631 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -1991,6 +1991,22 @@ object functions { def tan(columnName: String):

[GitHub] spark pull request #17981: [SPARK-15767][ML][SparkR] Decision Tree wrapper i...

2017-05-16 Thread bdwyer2
Github user bdwyer2 commented on a diff in the pull request: https://github.com/apache/spark/pull/17981#discussion_r116859197 --- Diff: R/pkg/R/mllib_tree.R --- @@ -499,3 +543,199 @@ setMethod("write.ml", signature(object = "RandomForestClassificationModel", path

[GitHub] spark pull request #17848: [SPARK-20586] [SQL] Add deterministic and distinc...

2017-05-16 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17848#discussion_r116857070 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/UDFRegistration.scala --- @@ -115,6 +115,22 @@ class UDFRegistration private[sql] (functionRegistry:

[GitHub] spark issue #17978: [SPARK-20736][Python] PySpark StringIndexer supports Str...

2017-05-16 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/17978 LGTM, ping @holdenk @jkbradley if they are interested --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #18003: [SparkR] Fix bad examples in DataFrame methods an...

2017-05-16 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18003#discussion_r116854470 --- Diff: R/pkg/R/DataFrame.R --- @@ -2814,7 +2815,7 @@ setMethod("except", #' path <- "path/to/file.json" #' df <- read.json(path) #'

[GitHub] spark pull request #18003: [SparkR] Fix bad examples in DataFrame methods an...

2017-05-16 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18003#discussion_r116853266 --- Diff: R/pkg/R/DataFrame.R --- @@ -2814,7 +2815,7 @@ setMethod("except", #' path <- "path/to/file.json" #' df <- read.json(path) #'

[GitHub] spark pull request #18003: [SparkR] Fix bad examples in DataFrame methods an...

2017-05-16 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18003#discussion_r116853922 --- Diff: R/pkg/R/functions.R --- @@ -54,7 +54,8 @@ setMethod("lit", signature("ANY"), #' @name abs #' @family normal_funcs #' @export

[GitHub] spark issue #17713: [SPARK-20417][SQL] Move subquery error handling to check...

2017-05-16 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/17713 ping @hvanhovell --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #17935: [SPARK-20690][SQL] Subqueries in FROM should have...

2017-05-16 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17935#discussion_r116851595 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -473,7 +473,7 @@ identifierComment

[GitHub] spark issue #17746: [SPARK-20449][ML] Upgrade breeze version to 0.13.1

2017-05-16 Thread superbobry
Github user superbobry commented on the issue: https://github.com/apache/spark/pull/17746 @srowen thanks! I've missed the point that 0.13.1 was intentionally merged into the upcoming release. @dbtsai could you give an example of the breaking API change between 0.12 and

[GitHub] spark issue #17990: [YARN] [SPARK-20756][WIP] yarn-shuffle jar references un...

2017-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17990 **[Test build #76982 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76982/testReport)** for PR 17990 at commit

[GitHub] spark issue #18006: [SQL][TRIVIAL] Lower parser log level to debug

2017-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18006 **[Test build #76981 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76981/testReport)** for PR 18006 at commit

[GitHub] spark pull request #18006: [SQL][TRIVIAL] Lower parser log level to debug

2017-05-16 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/18006 [SQL][TRIVIAL] Lower parser log level to debug ## What changes were proposed in this pull request? Currently the parser logs the query it is parsing at `info` level. This is too high, this

[GitHub] spark issue #17990: [YARN] [SPARK-20756][WIP] yarn-shuffle jar references un...

2017-05-16 Thread markgrover
Github user markgrover commented on the issue: https://github.com/apache/spark/pull/17990 Jenkins, re-test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #17990: [YARN] [SPARK-20756][WIP] yarn-shuffle jar refere...

2017-05-16 Thread markgrover
Github user markgrover commented on a diff in the pull request: https://github.com/apache/spark/pull/17990#discussion_r116845002 --- Diff: common/network-yarn/pom.xml --- @@ -113,6 +116,13 @@ io.netty.** +

[GitHub] spark issue #12646: [SPARK-14878][SQL] Trim characters string function suppo...

2017-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12646 **[Test build #76980 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76980/testReport)** for PR 12646 at commit

[GitHub] spark issue #16812: [SPARK-19465][SQL] Added options for custom boolean valu...

2017-05-16 Thread dhunziker
Github user dhunziker commented on the issue: https://github.com/apache/spark/pull/16812 That would remain a workaround though. The uniVocity parser for boolean supports this as well:

[GitHub] spark issue #17916: [SPARK-20590][SQL] Use Spark internal datasource if mult...

2017-05-16 Thread chrishfish
Github user chrishfish commented on the issue: https://github.com/apache/spark/pull/17916 Awesome @HyukjinKwon glad this issue has been resolved permanently 👍 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #17999: [SPARK-20751][SQL] Add built-in SQL Function - CO...

2017-05-16 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17999#discussion_r116842254 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -1991,6 +1991,22 @@ object functions { def tan(columnName: String):

[GitHub] spark pull request #17999: [SPARK-20751][SQL] Add built-in SQL Function - CO...

2017-05-16 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17999#discussion_r116842201 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -1991,6 +1991,22 @@ object functions { def tan(columnName: String):

[GitHub] spark issue #18005: [SPARK-20773][SQL] ParquetWriteSupport.writeFields is qu...

2017-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18005 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #15821: [SPARK-13534][PySpark] Using Apache Arrow to increase pe...

2017-05-16 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/15821 >@icexelloss , yes Arrow supports it but Spark stores timestamps is a different way which caused some complication. After talking with Holden, we agreed it was better to keep this PR to simple

[GitHub] spark pull request #18005: [SPARK-20773][SQL] ParquetWriteSupport.writeField...

2017-05-16 Thread tpoterba
GitHub user tpoterba opened a pull request: https://github.com/apache/spark/pull/18005 [SPARK-20773][SQL] ParquetWriteSupport.writeFields is quadratic in number of fields Fix quadratic List indexing in ParquetWriteSupport. I noticed this function while profiling some code

[GitHub] spark issue #17746: [SPARK-20449][ML] Upgrade breeze version to 0.13.1

2017-05-16 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17746 @superbobry see the discussion above? doesn't seem safe to do so --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #17746: [SPARK-20449][ML] Upgrade breeze version to 0.13.1

2017-05-16 Thread superbobry
Github user superbobry commented on the issue: https://github.com/apache/spark/pull/17746 Hello, are there any plans to backport this into 2.1 branch? The LBFGS and other fixex in 0.13.1 seem important enough. --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #18003: [SparkR] Fix bad examples in DataFrame methods and style...

2017-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18003 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76979/ Test PASSed. ---

[GitHub] spark issue #18003: [SparkR] Fix bad examples in DataFrame methods and style...

2017-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18003 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18003: [SparkR] Fix bad examples in DataFrame methods and style...

2017-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18003 **[Test build #76979 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76979/testReport)** for PR 18003 at commit

[GitHub] spark issue #17902: [SPARK-20641][core] Add key-value store abstraction and ...

2017-05-16 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/17902 Sorry I haven't had time to review this, given it's 100% new code I haven't have time to give it a proper read through. Hopefully I can find time by end of month (if not sooner) --- If your

[GitHub] spark issue #17400: [SPARK-19981][SQL] Update output partitioning info. in P...

2017-05-16 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/17400 ping --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #17758: [SPARK-20460][SQL] Make it more consistent to handle col...

2017-05-16 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/17758 ping --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #16989: [SPARK-19659] Fetch big blocks to disk when shuffle-read...

2017-05-16 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/16989 @jinxing64 Apologies for the delays in my response ... Can you take over this PR review @cloud-fan ? You have been doing the reviews way more than me on this anyway :-) Unfortunately I

[GitHub] spark pull request #16989: [SPARK-19659] Fetch big blocks to disk when shuff...

2017-05-16 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/16989#discussion_r116830282 --- Diff: core/src/main/scala/org/apache/spark/scheduler/MapStatus.scala --- @@ -193,8 +216,21 @@ private[spark] object HighlyCompressedMapStatus {

[GitHub] spark issue #18003: [SparkR] Fix bad examples in DataFrame methods and style...

2017-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18003 **[Test build #76979 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76979/testReport)** for PR 18003 at commit

[GitHub] spark issue #18003: [SparkR] Fix bad examples in DataFrame methods

2017-05-16 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/18003 @bdwyer2 Thanks for the suggestion. I ran a check on all the R scripts and fixed the style issues. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #17925: [SPARK-20205][core] Make sure StageInfo is updated befor...

2017-05-16 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/17925 Ping --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #17998: [SPARK-20703][SQL][WIP] Add an operator for writing data...

2017-05-16 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17998 How about `CreateDataSourceTableAsSelectCommand`? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #18002: [SPARK-20770][SQL] Improve ColumnStats

2017-05-16 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18002 @hvanhovell would it be possible to take a look? cc @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #17998: [SPARK-20703][SQL][WIP] Add an operator for writing data...

2017-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17998 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14741: [SPARK-6832][SPARKR][WIP]Handle partial reads in ...

2017-05-16 Thread krishnakalyan3
Github user krishnakalyan3 closed the pull request at: https://github.com/apache/spark/pull/14741 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #17998: [SPARK-20703][SQL][WIP] Add an operator for writing data...

2017-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17998 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76960/ Test PASSed. ---

[GitHub] spark issue #17998: [SPARK-20703][SQL][WIP] Add an operator for writing data...

2017-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17998 **[Test build #76960 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76960/testReport)** for PR 17998 at commit

[GitHub] spark pull request #18003: [SparkR] Fix bad examples in DataFrame methods

2017-05-16 Thread bdwyer2
Github user bdwyer2 commented on a diff in the pull request: https://github.com/apache/spark/pull/18003#discussion_r116820964 --- Diff: R/pkg/R/DataFrame.R --- @@ -3175,7 +3176,8 @@ setMethod("with", #' @aliases str,SparkDataFrame-method #' @family SparkDataFrame

[GitHub] spark issue #16648: [SPARK-18016][SQL][CATALYST] Code Generation: Constant P...

2017-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16648 **[Test build #76978 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76978/testReport)** for PR 16648 at commit

[GitHub] spark issue #18002: [SPARK-20770][SQL] Improve ColumnStats

2017-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18002 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18002: [SPARK-20770][SQL] Improve ColumnStats

2017-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18002 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76975/ Test PASSed. ---

[GitHub] spark issue #18002: [SPARK-20770][SQL] Improve ColumnStats

2017-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18002 **[Test build #76975 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76975/testReport)** for PR 18002 at commit

[GitHub] spark issue #17987: [SPARK-19707][SPARK-18922][TESTS][SQL][CORE] Fix test fa...

2017-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17987 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17987: [SPARK-19707][SPARK-18922][TESTS][SQL][CORE] Fix test fa...

2017-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17987 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76965/ Test PASSed. ---

[GitHub] spark pull request #17993: [SPARK-20758][SQL] Add Constant propagation optim...

2017-05-16 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17993#discussion_r116817636 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -54,6 +54,59 @@ object ConstantFolding extends

[GitHub] spark issue #17987: [SPARK-19707][SPARK-18922][TESTS][SQL][CORE] Fix test fa...

2017-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17987 **[Test build #76965 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76965/testReport)** for PR 17987 at commit

[GitHub] spark issue #16648: [SPARK-18016][SQL][CATALYST] Code Generation: Constant P...

2017-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16648 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76973/ Test PASSed. ---

[GitHub] spark issue #16648: [SPARK-18016][SQL][CATALYST] Code Generation: Constant P...

2017-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16648 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16648: [SPARK-18016][SQL][CATALYST] Code Generation: Constant P...

2017-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16648 **[Test build #76973 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76973/testReport)** for PR 16648 at commit

[GitHub] spark issue #17953: [SPARK-20680][SQL] Spark-sql do not support for void col...

2017-05-16 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17953 After this PR, we can describe it, but the query results are still empty. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #18004: [SPARK-18838][CORE] Introduce blocking strategy for Live...

2017-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18004 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #17999: [SPARK-20751][SQL] Add built-in SQL Function - COT

2017-05-16 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17999 Thanks for working on it. You need to add it to `FunctionRegistry` and add the SQL-related test cases to `operators.sql`. Also covers all the edge cases. For example, `0` that should trigger

[GitHub] spark pull request #18004: [SPARK-18838][CORE] Introduce blocking strategy f...

2017-05-16 Thread bOOm-X
GitHub user bOOm-X opened a pull request: https://github.com/apache/spark/pull/18004 [SPARK-18838][CORE] Introduce blocking strategy for LiveListener ## What changes were proposed in this pull request? When the queue of the LiveListener is full, events are dropped in an

[GitHub] spark pull request #17999: [SPARK-20751][SQL] Add built-in SQL Function - CO...

2017-05-16 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17999#discussion_r116812132 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -544,6 +544,28 @@ case class Sqrt(child:

[GitHub] spark pull request #17999: [SPARK-20751][SQL] Add built-in SQL Function - CO...

2017-05-16 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17999#discussion_r116811946 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -544,6 +544,28 @@ case class Sqrt(child:

[GitHub] spark issue #17899: [SPARK-20636] Add new optimization rule to flip adjacent...

2017-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17899 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17899: [SPARK-20636] Add new optimization rule to flip adjacent...

2017-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17899 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76970/ Test PASSed. ---

[GitHub] spark issue #17899: [SPARK-20636] Add new optimization rule to flip adjacent...

2017-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17899 **[Test build #76970 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76970/testReport)** for PR 17899 at commit

[GitHub] spark issue #18003: [SparkR] Fix bad examples in DataFrame methods

2017-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18003 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18003: [SparkR] Fix bad examples in DataFrame methods

2017-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18003 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76976/ Test PASSed. ---

[GitHub] spark issue #18003: [SparkR] Fix bad examples in DataFrame methods

2017-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18003 **[Test build #76976 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76976/testReport)** for PR 18003 at commit

[GitHub] spark issue #17087: [SPARK-19372][SQL] Fix throwing a Java exception at df.f...

2017-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17087 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17087: [SPARK-19372][SQL] Fix throwing a Java exception at df.f...

2017-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17087 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76972/ Test PASSed. ---

[GitHub] spark issue #17087: [SPARK-19372][SQL] Fix throwing a Java exception at df.f...

2017-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17087 **[Test build #76972 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76972/testReport)** for PR 17087 at commit

[GitHub] spark issue #17763: [SPARK-13747][Core]Add ThreadUtils.awaitReady and disall...

2017-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17763 **[Test build #76977 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76977/testReport)** for PR 17763 at commit

[GitHub] spark pull request #17821: [SPARK-20529][Core]Allow worker and master work w...

2017-05-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17821 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17763: [SPARK-13747][Core]Add ThreadUtils.awaitReady and disall...

2017-05-16 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/17763 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #17999: [SPARK-20751][SQL] Add built-in SQL Function - COT

2017-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17999 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76959/ Test PASSed. ---

[GitHub] spark issue #17999: [SPARK-20751][SQL] Add built-in SQL Function - COT

2017-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17999 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17999: [SPARK-20751][SQL] Add built-in SQL Function - COT

2017-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17999 **[Test build #76959 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76959/testReport)** for PR 17999 at commit

[GitHub] spark issue #17821: [SPARK-20529][Core]Allow worker and master work with a p...

2017-05-16 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/17821 Thanks! Merging to master and 2.2. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #18000: [SPARK-20364][SQL] Disable Parquet predicate pushdown fo...

2017-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18000 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76958/ Test PASSed. ---

[GitHub] spark issue #18000: [SPARK-20364][SQL] Disable Parquet predicate pushdown fo...

2017-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18000 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18000: [SPARK-20364][SQL] Disable Parquet predicate pushdown fo...

2017-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18000 **[Test build #76958 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76958/testReport)** for PR 18000 at commit

[GitHub] spark issue #18000: [SPARK-20364][SQL] Disable Parquet predicate pushdown fo...

2017-05-16 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18000 In addition, we need to know the limitation of column names in Parquet. See the related PR in Parquet: https://github.com/apache/parquet-mr/pull/361 --- If your project is set up for it, you

[GitHub] spark issue #17739: [SPARK-20443][MLLIB][ML] set ALS blockify size

2017-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17739 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76971/ Test FAILed. ---

[GitHub] spark issue #17739: [SPARK-20443][MLLIB][ML] set ALS blockify size

2017-05-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17739 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17739: [SPARK-20443][MLLIB][ML] set ALS blockify size

2017-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17739 **[Test build #76971 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76971/testReport)** for PR 17739 at commit

[GitHub] spark issue #18000: [SPARK-20364][SQL] Disable Parquet predicate pushdown fo...

2017-05-16 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18000 Now, the fix is much safer for merging to 2.2. Will review the test case later. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request #18000: [SPARK-20364][SQL] Disable Parquet predicate push...

2017-05-16 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18000#discussion_r116805464 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -168,6 +168,11 @@ private[parquet]

[GitHub] spark pull request #18000: [SPARK-20364][SQL] Disable Parquet predicate push...

2017-05-16 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18000#discussion_r116805363 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -168,6 +168,11 @@ private[parquet]

[GitHub] spark issue #15821: [SPARK-13534][PySpark] Using Apache Arrow to increase pe...

2017-05-16 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/15821 >@BryanCutler , is Timestamp and Date type supported now with Arrow 0.3? @icexelloss , yes Arrow supports it but Spark stores timestamps is a different way which caused some

[GitHub] spark issue #15821: [SPARK-13534][PySpark] Using Apache Arrow to increase pe...

2017-05-16 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/15821 No problem @rxin , I will restructure the tests so that the json data is local to each test, and ping you when done. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request #18000: [SPARK-20364][SQL] Disable Parquet predicate push...

2017-05-16 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18000#discussion_r116801592 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -168,6 +168,11 @@ private[parquet]

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-05-16 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r116801339 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -461,68 +462,249 @@ case class

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-05-16 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r116801396 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -461,68 +462,249 @@ case class

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-05-16 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r116801366 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -461,68 +462,249 @@ case class

[GitHub] spark pull request #12646: [SPARK-14878][SQL] Trim characters string functio...

2017-05-16 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/12646#discussion_r116801167 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -461,68 +462,249 @@ case class

[GitHub] spark issue #18003: [SparkR] Fix bad examples in DataFrame methods

2017-05-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18003 **[Test build #76976 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76976/testReport)** for PR 18003 at commit

<    1   2   3   4   5   >