[GitHub] spark issue #23108: [Spark-25993][SQL][TEST]Add test cases for CREATE EXTERN...

2018-12-06 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/23108 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...

2018-12-05 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22141 @maropu Very sorry. I haven't had the time to come back to it. I have some stuff on my plate. I will get to this after i am done. Thanks

[GitHub] spark issue #23108: [Spark-25993][SQL][TEST]Add test cases for CREATE EXTERN...

2018-12-05 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/23108 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #23211: [SPARK-19712][SQL] Move PullupCorrelatedPredicates and R...

2018-12-04 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/23211 @wangyum Thanks.. Can you please tell me how you generate this ? Also, is it possible to get runtimes of these queries to see if there are any regressions

[GitHub] spark issue #22899: [SPARK-25573] Combine resolveExpression and resolve in t...

2018-12-03 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22899 @gatorsmile Thanks a lot. I completely agree that we should try and combine these two. I will continue to think about

[GitHub] spark issue #23211: [SPARK-19712][SQL] Move PullupCorrelatedPredicates and R...

2018-12-03 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/23211 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #23211: [SPARK-19712][SQL] Move PullupCorrelatedPredicate...

2018-12-03 Thread dilipbiswal
GitHub user dilipbiswal opened a pull request: https://github.com/apache/spark/pull/23211 [SPARK-19712][SQL] Move PullupCorrelatedPredicates and RewritePredicateSubquery after OptimizeSubqueries Currently predicate subqueries (IN/EXISTS) are converted to Joins at the end

[GitHub] spark issue #23170: [SPARK-24423][FOLLOW-UP][SQL] Fix error example

2018-11-30 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/23170 Looks good to me. Thanks for fixing this @wangyum. I guess, i tried to trim off the example to show the mutual exclusivity of these two parameters in question

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-11-12 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r232571944 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Max.scala --- @@ -57,3 +57,34 @@ case class Max(child

[GitHub] spark issue #22899: [SPARK-25573] Combine resolveExpression and resolve in t...

2018-11-11 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22899 cc @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #22899: [SPARK-25573] Combine resolveExpression and resolve in t...

2018-11-10 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22899 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22788: [SPARK-25769][SQL]escape nested columns by backti...

2018-10-31 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22788#discussion_r229797016 --- Diff: sql/core/src/test/resources/sql-tests/results/columnresolution-negative.sql.out --- @@ -161,7 +161,7 @@ SELECT db1.t1.i1 FROM t1, mydb2.t1

[GitHub] spark issue #22899: [SPARK-25573] Combine resolveExpression and resolve in t...

2018-10-31 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22899 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22900: [SPARK-25618][SQL][TEST] Reduce time taken to execute Ka...

2018-10-31 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22900 Oh...thank you very much @dongjoon-hyun --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22900: [SPARK-25618][SQL][TEST] Reduce time taken to exe...

2018-10-31 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22900#discussion_r229625779 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaDontFailOnDataLossSuite.scala --- @@ -221,7 +221,7 @@ class

[GitHub] spark issue #22900: [SPARK-25618][SQL][TEST] Reduce time taken to execute Ka...

2018-10-31 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22900 cc @srowen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22900: [SPARK-25618][SQL][TEST] Reduce time taken to execute Ka...

2018-10-31 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22900 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22895: [SPARK-25886][SQL][Minor] Improve error message of `Fail...

2018-10-31 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22895 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #21860: [SPARK-24901][SQL]Merge the codegen of RegularHashMap an...

2018-10-31 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/21860 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-31 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22847 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22899: [SPARK-25573] Combine resolveExpression and resolve in t...

2018-10-31 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22899 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22788: [SPARK-25769][SQL]escape nested columns by backti...

2018-10-31 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22788#discussion_r229578738 --- Diff: sql/core/src/test/resources/sql-tests/results/columnresolution-negative.sql.out --- @@ -161,7 +161,7 @@ SELECT db1.t1.i1 FROM t1, mydb2.t1

[GitHub] spark pull request #22788: [SPARK-25769][SQL]escape nested columns by backti...

2018-10-31 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22788#discussion_r229577806 --- Diff: sql/core/src/test/resources/sql-tests/results/columnresolution-negative.sql.out --- @@ -161,7 +161,7 @@ SELECT db1.t1.i1 FROM t1, mydb2.t1

[GitHub] spark pull request #22900: [SPARK-25618][SQL][TEST] Reduce time taken to exe...

2018-10-31 Thread dilipbiswal
GitHub user dilipbiswal opened a pull request: https://github.com/apache/spark/pull/22900 [SPARK-25618][SQL][TEST] Reduce time taken to execute KafkaContinuousSourceStressForDontFailOnDataLossSuite ## What changes were proposed in this pull request? In this test, i have reduced

[GitHub] spark issue #22899: [SPARK-25573] Combine resolveExpression and resolve in t...

2018-10-31 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22899 cc @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #22899: [SPARK-25573] Combine resolveExpression and resol...

2018-10-31 Thread dilipbiswal
GitHub user dilipbiswal opened a pull request: https://github.com/apache/spark/pull/22899 [SPARK-25573] Combine resolveExpression and resolve in the Analyzer ## What changes were proposed in this pull request? Currently in the Analyzer, we have two methods 1) Resolve 2

[GitHub] spark issue #17520: [WIP][SPARK-19712][SQL] Move PullupCorrelatedPredicates ...

2018-10-30 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/17520 @gatorsmile Sure Sean. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22778: [SPARK-25784][SQL] Infer filters from constraints...

2018-10-30 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22778#discussion_r229362181 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -171,10 +171,13 @@ abstract class Optimizer

[GitHub] spark pull request #22778: [SPARK-25784][SQL] Infer filters from constraints...

2018-10-29 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22778#discussion_r229181821 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -171,10 +171,13 @@ abstract class Optimizer

[GitHub] spark pull request #22868: [SPARK-25833][SQL][DOCS] Update migration guide f...

2018-10-29 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22868#discussion_r229063489 --- Diff: docs/sql-migration-guide-hive-compatibility.md --- @@ -53,7 +53,20 @@ Spark SQL supports the vast majority of Hive features

[GitHub] spark pull request #22868: [SPARK-25833][SQL][DOCS] Update migration guide f...

2018-10-29 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22868#discussion_r229062732 --- Diff: docs/sql-migration-guide-hive-compatibility.md --- @@ -51,6 +51,22 @@ Spark SQL supports the vast majority of Hive features

[GitHub] spark pull request #22868: [SPARK-25833][SQL][DOCS] Update migration guide f...

2018-10-29 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22868#discussion_r229051205 --- Diff: docs/sql-migration-guide-hive-compatibility.md --- @@ -53,7 +53,20 @@ Spark SQL supports the vast majority of Hive features

[GitHub] spark issue #22856: [SPARK-25856][SQL][MINOR] Remove AverageLike and CountLi...

2018-10-29 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22856 Thank you @srowen @dongjoon-hyun @mgaido91 @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #22868: [SPARK-25833][SQL][DOCS] Update migration guide f...

2018-10-28 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22868#discussion_r228766091 --- Diff: docs/sql-migration-guide-hive-compatibility.md --- @@ -51,6 +51,9 @@ Spark SQL supports the vast majority of Hive features

[GitHub] spark pull request #22797: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-27 Thread dilipbiswal
Github user dilipbiswal closed the pull request at: https://github.com/apache/spark/pull/22797 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22809: [SPARK-19851][SQL] Add support for EVERY and ANY (SOME) ...

2018-10-27 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22809 Thanks a lot @cloud-fan @mgaido91 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22047: [SPARK-19851] Add support for EVERY and ANY (SOME...

2018-10-27 Thread dilipbiswal
Github user dilipbiswal closed the pull request at: https://github.com/apache/spark/pull/22047 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-27 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228725645 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/UnevaluableAggs.scala --- @@ -0,0 +1,62

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-26 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228666789 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/UnevaluableAggs.scala --- @@ -0,0 +1,62

[GitHub] spark pull request #22856: [SPARK-25856][SQL][MINOR] Remove AverageLike and ...

2018-10-26 Thread dilipbiswal
GitHub user dilipbiswal opened a pull request: https://github.com/apache/spark/pull/22856 [SPARK-25856][SQL][MINOR] Remove AverageLike and CountLike classes ## What changes were proposed in this pull request? These two classes were added for regr_ expression support (SPARK-23907

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-26 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228644504 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/UnevaluableAggs.scala --- @@ -0,0 +1,62

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-26 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228638310 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/UnevaluableAggs.scala --- @@ -0,0 +1,62

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-26 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228636110 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala --- @@ -282,6 +283,31 @@ trait RuntimeReplaceable

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-26 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228634827 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala --- @@ -282,6 +283,31 @@ trait RuntimeReplaceable

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-26 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228634159 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/UnevaluableAggs.scala --- @@ -0,0 +1,62

[GitHub] spark issue #22809: [SPARK-19851][SQL] Add support for EVERY and ANY (SOME) ...

2018-10-26 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22809 @cloud-fan @mgaido91 I have incorporated the comments. Could we please check if things look okay now ? --- - To unsubscribe

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-25 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228239274 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Max.scala --- @@ -57,3 +57,34 @@ case class Max(child

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-25 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228235105 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Max.scala --- @@ -57,3 +57,34 @@ case class Max(child

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-25 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228229829 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Max.scala --- @@ -57,3 +57,34 @@ case class Max(child

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-25 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228227624 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Max.scala --- @@ -57,3 +57,34 @@ case class Max(child

[GitHub] spark issue #22821: [SPARK-25832][SQL] remove newly added map related functi...

2018-10-25 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22821 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-24 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228020366 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala --- @@ -727,4 +728,67 @@ class DataFrameAggregateSuite extends

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-24 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r227901365 --- Diff: sql/core/src/test/resources/sql-tests/inputs/group-by.sql --- @@ -80,3 +80,69 @@ SELECT 1 FROM range(10) HAVING true; SELECT 1 FROM range

[GitHub] spark issue #22809: [SPARK-19851][SQL] Add support for EVERY and ANY (SOME) ...

2018-10-24 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22809 @cloud-fan Yeah.. I have some tests in group-by.sql. Please take a look. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-24 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r227881077 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/finishAnalysis.scala --- @@ -38,6 +39,18 @@ object ReplaceExpressions

[GitHub] spark issue #22809: [SPARK-19851][SQL] Add support for EVERY and ANY (SOME) ...

2018-10-23 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22809 cc @cloud-fan @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-23 Thread dilipbiswal
GitHub user dilipbiswal opened a pull request: https://github.com/apache/spark/pull/22809 [SPARK-19851][SQL] Add support for EVERY and ANY (SOME) aggregates ## What changes were proposed in this pull request? Implements Every, Some, Any aggregates in SQL. These new

[GitHub] spark issue #22797: [SPARK-19851][SQL] Add support for EVERY and ANY (SOME) ...

2018-10-23 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22797 @cloud-fan OK.. thanks a LOT. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #22797: [SPARK-19851][SQL] Add support for EVERY and ANY (SOME) ...

2018-10-23 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22797 @cloud-fan Sure.. Let me give that a try. Thanks --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22797: [SPARK-19851][SQL] Add support for EVERY and ANY (SOME) ...

2018-10-22 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22797 cc @cloud-fan @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #22797: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-22 Thread dilipbiswal
GitHub user dilipbiswal opened a pull request: https://github.com/apache/spark/pull/22797 [SPARK-19851][SQL] Add support for EVERY and ANY (SOME) aggregates ## What changes were proposed in this pull request? Implements Every, Some, Any aggregates in SQL. Logically

[GitHub] spark pull request #22047: [SPARK-19851] Add support for EVERY and ANY (SOME...

2018-10-22 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22047#discussion_r227054412 --- Diff: python/pyspark/sql/functions.py --- @@ -403,6 +403,28 @@ def countDistinct(col, *cols): return Column(jc) +def every

[GitHub] spark issue #22772: [SPARK-24499][SQL][DOC][Followup] Fix some broken links

2018-10-20 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22772 Thank you @gatorsmile @HyukjinKwon @xuanyuanking --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22772: [SPARK-24499][SQL][DOC][Followup] Fix some broken links

2018-10-18 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22772 @HyukjinKwon Sure.. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22772: [SPARK-24499][SQL][DOC][Followup] Fix some broken links

2018-10-18 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22772 @HyukjinKwon Sure. If there are more changes planned in another PR, then it can take these changes in. Since this went to 2.4, i thought there may be some urgency to fix the links

[GitHub] spark issue #22772: [SPARK-24499][SQL][DOC][Followup] Fix some broken links

2018-10-18 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22772 cc @dongjoon-hyun @gatorsmile @xuanyuanking --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #22772: [SPARK-24499][SQL][DOC][Followup] Fix some broken...

2018-10-18 Thread dilipbiswal
GitHub user dilipbiswal opened a pull request: https://github.com/apache/spark/pull/22772 [SPARK-24499][SQL][DOC][Followup] Fix some broken links ## What changes were proposed in this pull request? Fix some broken links in the new document. I have clicked through all the links

[GitHub] spark issue #22670: [SPARK-25631][SPARK-25632][SQL][TEST] Improve the test r...

2018-10-16 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22670 @srowen Thank you very much. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #22047: [SPARK-19851] Add support for EVERY and ANY (SOME...

2018-10-16 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22047#discussion_r225700828 --- Diff: python/pyspark/sql/functions.py --- @@ -403,6 +403,28 @@ def countDistinct(col, *cols): return Column(jc) +def every

[GitHub] spark issue #22670: [SPARK-25631][SPARK-25632][SQL][TEST] Improve the test r...

2018-10-12 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22670 @srowen OK, Let me look. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22670: [SPARK-25631][SPARK-25632][SQL][TEST] Improve the test r...

2018-10-12 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22670 @srowen Thanks. Did you mean, the test cases should extend a shared spark context (SharedKafkaSparkContext) which would have this property set ? Actually Sean, there are 3 suites

[GitHub] spark issue #22671: [SPARK-25615][SQL][TEST] Improve the test runtime of Kaf...

2018-10-11 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22671 Thanks a LOT @srowen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22641: [SPARK-25611][SPARK-25612][SQL][TESTS] Improve test run ...

2018-10-10 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22641 Thank you very much @srowen @fjh100456 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22671: [SPARK-25615][SQL][TEST] Improve the test runtime...

2018-10-09 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22671#discussion_r223819177 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSinkSuite.scala --- @@ -332,7 +332,9 @@ class KafkaSinkSuite

[GitHub] spark pull request #22671: [SPARK-25615][SQL][TEST] Improve the test runtime...

2018-10-09 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22671#discussion_r223771001 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSinkSuite.scala --- @@ -332,7 +332,9 @@ class KafkaSinkSuite

[GitHub] spark issue #22641: [SPARK-25611][SPARK-25612][SQL][TESTS] Improve test run ...

2018-10-09 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22641 @fjh100456 Yeah. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22671: [SPARK-25615][SQL][TEST] Improve the test runtime of Kaf...

2018-10-08 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22671 cc @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #22671: [SPARK-25615][SQL][TEST] Improve the test runtime...

2018-10-08 Thread dilipbiswal
GitHub user dilipbiswal opened a pull request: https://github.com/apache/spark/pull/22671 [SPARK-25615][SQL][TEST] Improve the test runtime of KafkaSinkSuite: streaming write to non-existing topic ## What changes were proposed in this pull request? Specify `kafka.max.block.ms

[GitHub] spark issue #22594: [SPARK-25674][SQL] If the records are incremented by mor...

2018-10-08 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22594 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22670: [SPARK-25631][SPARK-25632][SQL][TEST] Improve the test r...

2018-10-08 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22670 cc @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #22670: [SPARK-25631][SPARK-25632][SQL][TEST] Improve the...

2018-10-08 Thread dilipbiswal
GitHub user dilipbiswal opened a pull request: https://github.com/apache/spark/pull/22670 [SPARK-25631][SPARK-25632][SQL][TEST] Improve the test runtime of KafkaRDDSuite ## What changes were proposed in this pull request? Set a reasonable poll timeout thats used while consuming

[GitHub] spark issue #22641: [SPARK-25611][SPARK-25612][SQL][TESTS] Improve test run ...

2018-10-06 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22641 @srowen @gatorsmile Let me see if we can target a low hanging fruit first :-). Given the same input set is used for table and session codecs, we can skip when they both are same value

[GitHub] spark issue #22641: [SPARK-25611][SPARK-25612][SQL][TESTS] Improve test run ...

2018-10-06 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22641 @srowen > I would vote against running tests that we think have any value randomly. It's just the wrong way to solve problems, as much as it would be to simply run 90% of our test sui

[GitHub] spark issue #22641: [SPARK-25611][SPARK-25612][SQL][TESTS] Improve test run ...

2018-10-06 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22641 @srowen Thank you for your comments. Actually from a cursory look, i would agree that it does not look that pretty. I also agree that it does look like we are not testing as much as we used

[GitHub] spark issue #21732: [SPARK-24762][SQL] Enable Option of Product encoders

2018-10-06 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/21732 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22500: [SPARK-25488][SQL][TEST] Refactor MiscBenchmark to use m...

2018-10-06 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22500 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22448: [SPARK-25417][SQL] Improve findTightestCommonType to coe...

2018-10-06 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22448 @HyukjinKwon OK.. will do. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #22641: [SPARK-25611][SPARK-25612][SQL][TESTS] Improve test run ...

2018-10-05 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22641 @mgaido91 Thanks for your input. I took another look at the testcase. Let me outline some of my understandings first. - The test validates the precedence rules

[GitHub] spark issue #22648: [MINOR] Clean up the joinCriteria in SQL parser

2018-10-05 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22648 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22047: [SPARK-19851] Add support for EVERY and ANY (SOME...

2018-10-05 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22047#discussion_r223171963 --- Diff: python/pyspark/sql/functions.py --- @@ -403,6 +403,28 @@ def countDistinct(col, *cols): return Column(jc) +def every

[GitHub] spark issue #22638: [SPARK-25610][SQL][TEST] Improve execution time of Datas...

2018-10-05 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22638 Thanks a lot @gatorsmile @mgaido91 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22644: [SPARK-25626][SQL][TEST] Improve the test execution time...

2018-10-05 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22644 Thank you very much @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark issue #22644: [SPARK-25626][SQL][TEST] Improve the test execution time...

2018-10-05 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22644 cc @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #22642: [SPARK-25613][SPARK-25614][TEST] Add tag ExtendedHiveTes...

2018-10-05 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22642 @dongjoon-hyun Ah.. got it.. thanks a lot. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22644: [SPARK-25626] Improve the test execution time of ...

2018-10-05 Thread dilipbiswal
GitHub user dilipbiswal opened a pull request: https://github.com/apache/spark/pull/22644 [SPARK-25626] Improve the test execution time of HiveClientSuites ## What changes were proposed in this pull request? Improve the runtime by reducing the number of partitions created

[GitHub] spark issue #22642: [SPARK-25613][SPARK-25614][TEST] Add tag ExtendedHiveTes...

2018-10-05 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22642 @gengliangwang I have a question. What happens when we place this tag ? It is skipped on regular runs

[GitHub] spark issue #22047: [SPARK-19851] Add support for EVERY and ANY (SOME) aggre...

2018-10-05 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22047 @cloud-fan please see my comment [link](https://github.com/apache/spark/pull/22047#issuecomment-411536039). I had tried to rewrite using max and min as suggested by Herman and Reynold

[GitHub] spark issue #22641: [SPARK-25611][SPARK-25612][SQL][TESTS] Improve test run ...

2018-10-05 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22641 @mgaido91 Trying to understand the concern regarding "working combination and a non-working " comment. In my understanding, originally we were doing a cross join between two sets of

[GitHub] spark issue #22641: [SPARK-25611][SPARK-25622][SQL][TESTS] Improve test run ...

2018-10-05 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22641 @mgaido91 Could you please take a look ? Thank you. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #22641: [SPARK-25611][SPARK-25622][SQL][TESTS] Improve te...

2018-10-05 Thread dilipbiswal
GitHub user dilipbiswal opened a pull request: https://github.com/apache/spark/pull/22641 [SPARK-25611][SPARK-25622][SQL][TESTS] Improve test run time of CompressionCodecSuite ## What changes were proposed in this pull request? I am using the idea of @mgaido91 to pick the test

  1   2   3   4   5   6   7   8   9   10   >