[GitHub] spark issue #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-08 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17541 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the f

[GitHub] spark issue #17556: [SPARK-16957][MLlib] Use weighted midpoints for split va...

2017-04-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17556 **[Test build #3652 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3652/testReport)** for PR 17556 at commit [`9ca5750`](https://github.com/apache/spark/commit/9

[GitHub] spark pull request #17565: [MINOR] Issue: Change "slice" vs "partition" in e...

2017-04-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17565 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #17565: [MINOR] Issue: Change "slice" vs "partition" in exceptio...

2017-04-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17565 **[Test build #3650 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3650/testReport)** for PR 17565 at commit [`844df10`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #17575: [SPARK-20265][MLlib] Improve Prefix'span pre-processing ...

2017-04-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17575 **[Test build #3651 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3651/testReport)** for PR 17575 at commit [`8e5db6a`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #17572: [SPARK-20260][MLLib] String interpolation required for e...

2017-04-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17572 **[Test build #3649 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3649/testReport)** for PR 17572 at commit [`89aeb62`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #17565: [MINOR] Issue: Change "slice" vs "partition" in exceptio...

2017-04-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17565 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or i

[GitHub] spark pull request #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17541#discussion_r110533603 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveTableScanExec.scala --- @@ -198,18 +200,13 @@ case class HiveTableScanExec(

[GitHub] spark pull request #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17541#discussion_r110533216 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/LogicalRelation.scala --- @@ -43,17 +43,8 @@ case class LogicalRelation(

[GitHub] spark issue #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks of a re...

2017-04-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17569 **[Test build #75628 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75628/testReport)** for PR 17569 at commit [`10cf4be`](https://github.com/apache/spark/commit/10

[GitHub] spark issue #17568: [SPARK-20254][SQL] Remove unnecessary data conversion fo...

2017-04-08 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/17568 @cloud-fan how about this check for 2.? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark issue #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks of a re...

2017-04-08 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17569 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featur

[GitHub] spark pull request #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks ...

2017-04-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17569#discussion_r110530705 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -225,25 +225,26 @@ case class Invoke(

[GitHub] spark issue #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks of a re...

2017-04-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17569 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fea

[GitHub] spark pull request #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks ...

2017-04-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17569#discussion_r110530587 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -225,25 +225,26 @@ case class Invoke(

[GitHub] spark pull request #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks ...

2017-04-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17569#discussion_r110530518 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/RowEncoder.scala --- @@ -262,17 +264,18 @@ object RowEncoder {

[GitHub] spark pull request #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17541#discussion_r110530462 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -359,9 +359,59 @@ abstract class QueryPlan[PlanType <:

[GitHub] spark issue #17572: [SPARK-20260][MLLib] String interpolation required for e...

2017-04-08 Thread vijaykramesh
Github user vijaykramesh commented on the issue: https://github.com/apache/spark/pull/17572 @srowen fixed it in some more places. it seems like everywhere else that regexp matches we actually want the $ outputted. do you want me to squash commits as well? --- If your project is se

[GitHub] spark pull request #17576: Update Dataset to camel case (DataSet) to match D...

2017-04-08 Thread kevinmcinerney
Github user kevinmcinerney closed the pull request at: https://github.com/apache/spark/pull/17576 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featur

[GitHub] spark issue #17546: [SPARK-20233] [SQL] Apply star-join filter heuristics to...

2017-04-08 Thread ioana-delaney
Github user ioana-delaney commented on the issue: https://github.com/apache/spark/pull/17546 @cloud-fan Do you have any comments? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17575: [SPARK-20265][MLlib] Improve Prefix'span pre-processing ...

2017-04-08 Thread Syrux
Github user Syrux commented on the issue: https://github.com/apache/spark/pull/17575 Yo Sean, I already pushed the requested changes in case it's the correct place to do so. (I can just revert them, if not) I added two new methods to allow tests. First a method which finds

[GitHub] spark issue #15899: [SPARK-18466] added withFilter method to RDD

2017-04-08 Thread reggert
Github user reggert commented on the issue: https://github.com/apache/spark/pull/15899 Strictly speaking, this doesn't just affect pair RDDs. It affects any RDDs on which a `for` expression involving a filter operation, which includes explicit `if` clauses as well as pattern matches.

[GitHub] spark pull request #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17541#discussion_r110524716 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -267,7 +265,7 @@ case class FileSourceScanExec(

[GitHub] spark pull request #17540: [SPARK-20213][SQL][UI] Fix DataFrameWriter operat...

2017-04-08 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/17540#discussion_r110523942 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -180,9 +180,13 @@ class Dataset[T] private[sql]( // to happen right away

[GitHub] spark issue #17540: [SPARK-20213][SQL][UI] Fix DataFrameWriter operations in...

2017-04-08 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/17540 Thanks for the review! I'll get the thrift-server tests fixed up next week. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17541#discussion_r110523770 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -359,9 +359,59 @@ abstract class QueryPlan[PlanType <:

[GitHub] spark issue #17576: Update Dataset to camel case (DataSet) to match DataFram...

2017-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17576 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #17576: Update Dataset to camel case (DataSet) to match D...

2017-04-08 Thread kevinmcinerney
GitHub user kevinmcinerney opened a pull request: https://github.com/apache/spark/pull/17576 Update Dataset to camel case (DataSet) to match DataFrames Shouldn't Datasets and DataFrames both be camel case for the ocd ppl out there? ## What changes were proposed in this pull

[GitHub] spark pull request #16820: [SPARK-19471] AggregationIterator does not initia...

2017-04-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16820#discussion_r110523513 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala --- @@ -448,6 +448,22 @@ class DataFrameFunctionsSuite extends Query

[GitHub] spark pull request #15567: [SPARK-14393][SQL] values generated by non-determ...

2017-04-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15567#discussion_r110523451 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -205,10 +206,11 @@ case class FilterExec(condition:

[GitHub] spark pull request #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17541#discussion_r110523418 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -359,9 +359,59 @@ abstract class QueryPlan[PlanType <:

[GitHub] spark issue #17350: [SPARK-20017][SQL] change the nullability of function 'S...

2017-04-08 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17350 Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks of a re...

2017-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17569 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks of a re...

2017-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17569 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75627/ Test PASSed. ---

[GitHub] spark issue #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks of a re...

2017-04-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17569 **[Test build #75627 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75627/testReport)** for PR 17569 at commit [`510fb53`](https://github.com/apache/spark/commit/5

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-08 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110522547 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -54,14 +54,12 @@ case class CostBased

[GitHub] spark issue #17575: [SPARK-20265][MLlib] Improve Prefix'span pre-processing ...

2017-04-08 Thread Syrux
Github user Syrux commented on the issue: https://github.com/apache/spark/pull/17575 Ok, should I create a new Jira and push there the additionnal tests ? Or is here completly fine, since it's related to the current change Tell me, and I will get the change done asap :) -

[GitHub] spark issue #17350: [SPARK-20017][SQL] change the nullability of function 'S...

2017-04-08 Thread zhaorongsheng
Github user zhaorongsheng commented on the issue: https://github.com/apache/spark/pull/17350 @gatorsmile Sorry for the late reply. I have checked all the functions' nullability setting and I didn't found any issue. Thanks~ --- If your project is set up for it, you can reply

[GitHub] spark issue #17575: [SPARK-20265][MLlib] Improve Prefix'span pre-processing ...

2017-04-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17575 Even a simplistic test of this case would give a lot more confidence that it's correct. If it means opening up a `private[spark]` method or two to make testing possible that seems reasonable. I don'

[GitHub] spark pull request #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks ...

2017-04-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/17569#discussion_r110520996 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -225,25 +225,26 @@ case class Invoke(

[GitHub] spark issue #17574: [SPARK-20264][SQL] asm should be non-test dependency in ...

2017-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17574 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17574: [SPARK-20264][SQL] asm should be non-test dependency in ...

2017-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17574 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75626/ Test PASSed. ---

[GitHub] spark issue #17574: [SPARK-20264][SQL] asm should be non-test dependency in ...

2017-04-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17574 **[Test build #75626 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75626/testReport)** for PR 17574 at commit [`2a03188`](https://github.com/apache/spark/commit/2

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-08 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110520685 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -736,6 +736,12 @@ object SQLConf { .checkValue(weight => w

[GitHub] spark issue #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks of a re...

2017-04-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17569 **[Test build #75627 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75627/testReport)** for PR 17569 at commit [`510fb53`](https://github.com/apache/spark/commit/51

[GitHub] spark issue #17575: [SPARK-20265][MLlib] Improve Prefix'span pre-processing ...

2017-04-08 Thread Syrux
Github user Syrux commented on the issue: https://github.com/apache/spark/pull/17575 Yes exactly, the current implementation adds too much unnecessary delimiters. We this one line change, delimiter are only placed where needed. Currently there are no tests to verify if the al

[GitHub] spark pull request #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17541#discussion_r110519122 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -423,8 +423,15 @@ case class CatalogRelation(

[GitHub] spark issue #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17541 cc @gatorsmile any more comments? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks of a re...

2017-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17569 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks of a re...

2017-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17569 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75625/ Test PASSed. ---

[GitHub] spark issue #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks of a re...

2017-04-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17569 **[Test build #75625 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75625/testReport)** for PR 17569 at commit [`3080ac2`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #17568: [SPARK-20254][SQL] Remove unnecessary data conversion fo...

2017-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17568 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75624/ Test PASSed. ---

[GitHub] spark issue #17568: [SPARK-20254][SQL] Remove unnecessary data conversion fo...

2017-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17568 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17568: [SPARK-20254][SQL] Remove unnecessary data conversion fo...

2017-04-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17568 **[Test build #75624 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75624/testReport)** for PR 17568 at commit [`0679ebe`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #17242: [SPARK-19902][SQL] Add optimization rule to simplify exp...

2017-04-08 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17242 ping @cloud-fan Can you take a look of this? If you don't think this is appropriate direction, please let me know. --- If your project is set up for it, you can reply to this email and have your rep

[GitHub] spark issue #17574: [SPARK-20264][SQL] asm should be non-test dependency in ...

2017-04-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17574 **[Test build #75626 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75626/testReport)** for PR 17574 at commit [`2a03188`](https://github.com/apache/spark/commit/2a

[GitHub] spark pull request #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks ...

2017-04-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17569#discussion_r110517840 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -225,25 +225,26 @@ case class Invoke(

[GitHub] spark issue #17574: [SPARK-20264][SQL] asm should be non-test dependency in ...

2017-04-08 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17574 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, o

[GitHub] spark issue #17574: [SPARK-20264][SQL] asm should be non-test dependency in ...

2017-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17574 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17574: [SPARK-20264][SQL] asm should be non-test dependency in ...

2017-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17574 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75623/ Test FAILed. ---

[GitHub] spark issue #17574: [SPARK-20264][SQL] asm should be non-test dependency in ...

2017-04-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17574 **[Test build #75623 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75623/testReport)** for PR 17574 at commit [`2a03188`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #14731: [SPARK-17159] [streaming]: optimise check for new files ...

2017-04-08 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/14731 @srowen anything else I need to do here? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #12004: [SPARK-7481] [build] Add spark-cloud module to pull in o...

2017-04-08 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/12004 @srowen anything else I need to do here? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #17342: [SPARK-12868][SQL] Allow adding jars from hdfs

2017-04-08 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/17342#discussion_r110517523 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2767,3 +2767,24 @@ private[spark] class CircularBuffer(sizeInBytes: Int = 1024

[GitHub] spark issue #17467: [SPARK-20140][DStream] Remove hardcoded kinesis retry wa...

2017-04-08 Thread yssharma
Github user yssharma commented on the issue: https://github.com/apache/spark/pull/17467 Just for info, while trying to use the `sc` in the `KinesisBackedBlockRDD ` : `- Basic reading from Kinesis *** FAILED *** org.apache.spark.SparkException: Task not serializable

[GitHub] spark issue #17506: [SPARK-20189][DStream] Fix spark kinesis testcases to re...

2017-04-08 Thread yssharma
Github user yssharma commented on the issue: https://github.com/apache/spark/pull/17506 Is there anything else that can be done on this patch. The patch fixes all the deprecated api testcases that try to use the aws secret/id credentials instead of the builder. --- If your project i

[GitHub] spark issue #17467: [SPARK-20140][DStream] Remove hardcoded kinesis retry wa...

2017-04-08 Thread yssharma
Github user yssharma commented on the issue: https://github.com/apache/spark/pull/17467 @brkyvz - thanks for taking time to review the patch. appreciate it. Implemented all your suggestions. Now passing a new map for the kinesis configs and added mechanism to use the builder for th

[GitHub] spark issue #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks of a re...

2017-04-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17569 **[Test build #75625 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75625/testReport)** for PR 17569 at commit [`3080ac2`](https://github.com/apache/spark/commit/30

[GitHub] spark pull request #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks ...

2017-04-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/17569#discussion_r110516074 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/RowEncoder.scala --- @@ -262,17 +264,18 @@ object RowEncoder { inpu

[GitHub] spark issue #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17541 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75620/ Test PASSed. ---

[GitHub] spark issue #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17541 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17541 **[Test build #75620 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75620/testReport)** for PR 17541 at commit [`9305187`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #17568: [SPARK-20254][SQL] Remove unnecessary data conversion fo...

2017-04-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17568 **[Test build #75624 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75624/testReport)** for PR 17568 at commit [`0679ebe`](https://github.com/apache/spark/commit/06

[GitHub] spark issue #17574: [SPARK-20264][SQL] asm should be non-test dependency in ...

2017-04-08 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17574 Do we need it as a normal dependency? Looks like sql/core doesn't use it and the building works without this dependency. Sorry if I am missing something. --- If your project is set up for it, you ca

[GitHub] spark issue #17575: [SPARK-20265][MLlib] Improve Prefix'span pre-processing ...

2017-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17575 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #17575: [SPARK-20265][MLlib] Improve Prefix'span pre-proc...

2017-04-08 Thread Syrux
GitHub user Syrux opened a pull request: https://github.com/apache/spark/pull/17575 [SPARK-20265][MLlib] Improve Prefix'span pre-processing efficiency ## What changes were proposed in this pull request? Improve PrefixSpan pre-processing efficency by preventing sequences of z

[GitHub] spark issue #17574: [SPARK-20264][SQL] asm should be non-test dependency in ...

2017-04-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17574 **[Test build #75623 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75623/testReport)** for PR 17574 at commit [`2a03188`](https://github.com/apache/spark/commit/2a

[GitHub] spark issue #17574: [SPARK-20264][SQL] asm should be non-test dependency in ...

2017-04-08 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17574 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, o

[GitHub] spark issue #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks of a re...

2017-04-08 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17569 Seems there are places (i.e., `RowEncoder`) calling `isNullAt` which gives `returnNullable` as true (default value). --- If your project is set up for it, you can reply to this email and have your r

[GitHub] spark issue #17567: [SPARK-19991][CORE][YARN] FileSegmentManagedBuffer perfo...

2017-04-08 Thread witgo
Github user witgo commented on the issue: https://github.com/apache/spark/pull/17567 OK, I see. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks of a re...

2017-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17569 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75621/ Test FAILed. ---

[GitHub] spark issue #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks of a re...

2017-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17569 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks of a re...

2017-04-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17569 **[Test build #75621 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75621/testReport)** for PR 17569 at commit [`a39803a`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #17469: [SPARK-20132][Docs] Add documentation for column string ...

2017-04-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17469 **[Test build #75622 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75622/testReport)** for PR 17469 at commit [`bc03f3c`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #17469: [SPARK-20132][Docs] Add documentation for column string ...

2017-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17469 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75622/ Test FAILed. ---

[GitHub] spark issue #17469: [SPARK-20132][Docs] Add documentation for column string ...

2017-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17469 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17469: [SPARK-20132][Docs] Add documentation for column string ...

2017-04-08 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/17469 Great! I'll still follow up with Shane & Josh re: @felixcheung triggering build as well. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as we

[GitHub] spark issue #17469: [SPARK-20132][Docs] Add documentation for column string ...

2017-04-08 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17469 Yes, it seems from your comment @holdenk. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featu

[GitHub] spark issue #17469: [SPARK-20132][Docs] Add documentation for column string ...

2017-04-08 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17469 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes s

[GitHub] spark issue #17469: [SPARK-20132][Docs] Add documentation for column string ...

2017-04-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17469 **[Test build #75622 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75622/testReport)** for PR 17469 at commit [`bc03f3c`](https://github.com/apache/spark/commit/bc

[GitHub] spark issue #17469: [SPARK-20132][Docs] Add documentation for column string ...

2017-04-08 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17469 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes s

[GitHub] spark issue #17469: [SPARK-20132][Docs] Add documentation for column string ...

2017-04-08 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/17469 I've e-mailed them since the Jenkins configuration is a bit too involved (and I'd need Shane to sign off on any Jenkins change anyways). Sorry this is slowing down your PR @map222 and thank you so m

[GitHub] spark issue #17574: [SPARK-20264][SQL] asm should be non-test dependency in ...

2017-04-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17574 **[Test build #3648 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3648/testReport)** for PR 17574 at commit [`2a03188`](https://github.com/apache/spark/commit/

[GitHub] spark issue #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks of a re...

2017-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17569 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75619/ Test FAILed. ---

[GitHub] spark issue #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks of a re...

2017-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17569 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17469: [SPARK-20132][Docs] Add documentation for column string ...

2017-04-08 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/17469 Jenkins retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wish

[GitHub] spark issue #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks of a re...

2017-04-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17569 **[Test build #75619 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75619/testReport)** for PR 17569 at commit [`fc6caac`](https://github.com/apache/spark/commit/f

[GitHub] spark pull request #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks ...

2017-04-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/17569#discussion_r110513988 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetPrimitiveSuite.scala --- @@ -96,6 +96,16 @@ class DatasetPrimitiveSuite extends QueryTest with

[GitHub] spark pull request #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks ...

2017-04-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17569#discussion_r110513970 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/RowEncoder.scala --- @@ -262,17 +264,18 @@ object RowEncoder {

[GitHub] spark pull request #17569: [SPARK-20253][SQL] Remove unnecessary nullchecks ...

2017-04-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17569#discussion_r110513952 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetPrimitiveSuite.scala --- @@ -96,6 +96,16 @@ class DatasetPrimitiveSuite extends QueryTest wi

  1   2   >