[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-10-15 Thread raghavgautam
Github user raghavgautam commented on the issue: https://github.com/apache/spark/pull/22414 @srowen Ok. I will close this PR and the bug. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-10-15 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22414 ... look at the test failure at

[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-10-15 Thread raghavgautam
Github user raghavgautam commented on the issue: https://github.com/apache/spark/pull/22414 @srowen Can you give examples ? From what I have seen, invalid values because of typo say `1 sec` throw exception at groupby(). ---

[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-10-15 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22414 I get it, but this becomes inconsistent, right? other invalid window values aren't handled the same way. --- - To unsubscribe,

[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-10-15 Thread raghavgautam
Github user raghavgautam commented on the issue: https://github.com/apache/spark/pull/22414 @srowen Key benefit is fail fast particularly when running from spark-shell. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-10-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22414 Yeah, the test that failed here asserts that it's an `AnalysisException`. I guess it could be removed. The thing is, many other cases are still handled as `AnalysisException`. Maybe it's best to

[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22414 **[Test build #4374 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4374/testReport)** for PR 22414 at commit

[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22414 **[Test build #4374 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4374/testReport)** for PR 22414 at commit

[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-10-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22414 @raghavgautam Can you update this pr based on my patch? (sorry, but I have no time to take this over...) --- - To unsubscribe,

[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-10-01 Thread raghavgautam
Github user raghavgautam commented on the issue: https://github.com/apache/spark/pull/22414 @maropu Can you take a look ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-09-19 Thread raghavgautam
Github user raghavgautam commented on the issue: https://github.com/apache/spark/pull/22414 The problem is that the error message that we start getting is: ``` requirement failed: The window duration must be positive, but found: -12096 ``` Where as the error

[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-09-18 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22414 How about this? https://github.com/apache/spark/compare/master...maropu:pr22414 IMO simple fixes and tests are better. --- -

[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-09-18 Thread raghavgautam
Github user raghavgautam commented on the issue: https://github.com/apache/spark/pull/22414 @maropu Please continue with the review. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-09-17 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22414 Plz do not change the format: `Problem` -> `What changes were proposed in this pull request?`. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-09-17 Thread raghavgautam
Github user raghavgautam commented on the issue: https://github.com/apache/spark/pull/22414 @maropu I have updated the PR description and answered your question, can you continue with your review. --- - To

[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-09-15 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22414 Please write the current behaviour (you described above) in the PR description? Also, can you format the PR description, too? you don't need the link to the jira. ``` …alues should

[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-09-14 Thread raghavgautam
Github user raghavgautam commented on the issue: https://github.com/apache/spark/pull/22414 @maropu Can you continue with the review ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-09-14 Thread raghavgautam
Github user raghavgautam commented on the issue: https://github.com/apache/spark/pull/22414 The code below throws following error. Note that the error is produced at the time of count() call instead of window() call. ```java val df = spark.readStream .format("rate")

[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-09-13 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22414 btw, what's the current behaivour of Spark in case of the nagative value? --- - To unsubscribe, e-mail:

[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-09-13 Thread raghavgautam
Github user raghavgautam commented on the issue: https://github.com/apache/spark/pull/22414 @tdas Can you please take a look ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22414 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22414 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22414 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional