[GitHub] spark issue #22426: [SPARK-25436] Bump master branch version to 2.5.0-SNAPSH...

2018-09-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22426 **[Test build #96088 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96088/testReport)** for PR 22426 at commit

[GitHub] spark issue #22426: [SPARK-25436] Bump master branch version to 2.5.0-SNAPSH...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22426 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22426: [SPARK-25436] Bump master branch version to 2.5.0-SNAPSH...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22426 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22426: [SPARK-25436] Bump master branch version to 2.5.0...

2018-09-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22426#discussion_r217876458 --- Diff: docs/_config.yml --- @@ -14,8 +14,8 @@ include: # These allow the documentation to be updated with newer releases # of Spark,

[GitHub] spark pull request #22426: [SPARK-25436] Bump master branch version to 2.5.0...

2018-09-14 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/22426#discussion_r217876251 --- Diff: docs/_config.yml --- @@ -14,8 +14,8 @@ include: # These allow the documentation to be updated with newer releases # of Spark,

[GitHub] spark pull request #22010: [SPARK-21436][CORE] Take advantage of known parti...

2018-09-14 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/22010#discussion_r217876215 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -396,7 +396,26 @@ abstract class RDD[T: ClassTag]( * Return a new RDD containing

[GitHub] spark pull request #22010: [SPARK-21436][CORE] Take advantage of known parti...

2018-09-14 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/22010#discussion_r217876184 --- Diff: core/src/main/scala/org/apache/spark/rdd/MapPartitionsRDD.scala --- @@ -35,16 +35,24 @@ import org.apache.spark.{Partition, TaskContext} *

[GitHub] spark issue #22426: [SPARK-25436] Bump master branch version to 2.5.0-SNAPSH...

2018-09-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22426 cc @yhuai @rxin @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #22426: [SPARK-25436] Bump master branch version to 2.5.0-SNAPSH...

2018-09-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22426 **[Test build #96087 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96087/testReport)** for PR 22426 at commit

[GitHub] spark issue #22426: [SPARK-25436] Bump master branch version to 2.5.0-SNAPSH...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22426 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22426: [SPARK-25436] Bump master branch version to 2.5.0-SNAPSH...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22426 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #22426: [SPARK-25436] Bump master branch version to 2.5.0...

2018-09-14 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/22426 [SPARK-25436] Bump master branch version to 2.5.0-SNAPSHOT ## What changes were proposed in this pull request? In the dev list, we can still discuss whether the next version is 2.5.0 or

[GitHub] spark issue #22421: [SPARK-25431][SQL][EXAMPLES] Fix function examples and u...

2018-09-14 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22421 @dongjoon-hyun Sure, reverted. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21654: [SPARK-24671][PySpark] DataFrame length using a d...

2018-09-14 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21654#discussion_r217874544 --- Diff: python/pyspark/sql/dataframe.py --- @@ -375,6 +375,9 @@ def _truncate(self): return int(self.sql_ctx.getConf(

[GitHub] spark issue #20418: [SPARK-790][MESOS] Implement reregistered callback for M...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20418 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20418: [SPARK-790][MESOS] Implement reregistered callback for M...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20418 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22425: [SPARK-23367][Build] Include python document style check...

2018-09-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22425 **[Test build #96086 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96086/testReport)** for PR 22425 at commit

[GitHub] spark issue #22425: [SPARK-23367][Build] Include python document style check...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22425 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22425: [SPARK-23367][Build] Include python document style check...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22425 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #20556: [SPARK-23367][Build] Include python document style check...

2018-09-14 Thread rekhajoshm
Github user rekhajoshm commented on the issue: https://github.com/apache/spark/pull/20556 PR #22425 Closing this.Thanks --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20556: [SPARK-23367][Build] Include python document styl...

2018-09-14 Thread rekhajoshm
Github user rekhajoshm closed the pull request at: https://github.com/apache/spark/pull/20556 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22425: [SPARK-23367][Build] Include python document styl...

2018-09-14 Thread rekhajoshm
GitHub user rekhajoshm opened a pull request: https://github.com/apache/spark/pull/22425 [SPARK-23367][Build] Include python document style checking ## What changes were proposed in this pull request? Includes python document style checking. - Use sphinx like check, run

[GitHub] spark issue #22421: [SPARK-25431][SQL][EXAMPLES] Fix function examples and u...

2018-09-14 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22421 @ueshin . Could you revert the current one and send a new one please? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20556: [SPARK-23367][Build] Include python document style check...

2018-09-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20556 **[Test build #96085 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96085/testReport)** for PR 20556 at commit

[GitHub] spark issue #20556: [SPARK-23367][Build] Include python document style check...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20556 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20556: [SPARK-23367][Build] Include python document style check...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20556 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22421: [SPARK-25431][SQL][EXAMPLES] Fix function examples and u...

2018-09-14 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22421 @dongjoon-hyun I picked the string from `sql(...).show(truncate = false)` and modified a little. I'm sorry but I didn't notice that we use the `spark-sql` to pick the result. Let me send a follow-up

[GitHub] spark pull request #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycode...

2018-09-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22231 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...

2018-09-14 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22231 Merged to master/2.4 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22417: [SPARK-25426][SQL] Remove the duplicate fallback logic i...

2018-09-14 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/22417 LGTM too. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22231 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96082/ Test PASSed. ---

[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22231 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...

2018-09-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22231 **[Test build #96082 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96082/testReport)** for PR 22231 at commit

[GitHub] spark issue #22424: [SPARK-25303][STREAMING] For checkpointed DStreams, remo...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22424 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22423: [SPARK-25302][STREAMING] Checkpoint the reducedStream in...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22423 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22424: [SPARK-25303][STREAMING] For checkpointed DStreams, remo...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22424 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22424: [SPARK-25303][STREAMING] For checkpointed DStreams, remo...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22424 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22423: [SPARK-25302][STREAMING] Checkpoint the reducedStream in...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22423 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22424: [SPARK-25303][STREAMING] For checkpointed DStream...

2018-09-14 Thread nikunjb
GitHub user nikunjb opened a pull request: https://github.com/apache/spark/pull/22424 [SPARK-25303][STREAMING] For checkpointed DStreams, remove the parent… …RemeberDuration - this will be recalculated to a minimum value ## How was this patch tested?

[GitHub] spark issue #22423: [SPARK-25302][STREAMING] Checkpoint the reducedStream in...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22423 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22423: [SPARK-25302][STREAMING] Checkpoint the reducedSt...

2018-09-14 Thread nikunjb
GitHub user nikunjb opened a pull request: https://github.com/apache/spark/pull/22423 [SPARK-25302][STREAMING] Checkpoint the reducedStream in ReducedWindo… …wDStream so as to cut the lineage completely to parent ## What changes were proposed in this pull request?

[GitHub] spark pull request #22355: [SPARK-25358][SQL] MutableProjection supports fal...

2018-09-14 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22355#discussion_r217860646 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/InterpretedMutableProjection.scala --- @@ -0,0 +1,83 @@ +/* + *

[GitHub] spark issue #21677: [SPARK-24692][TESTS] Improvement FilterPushdownBenchmark

2018-09-14 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21677 @wangyum . If you are still interested in reverting your PR as you mentioned in [SPARK-25339](https://issues.apache.org/jira/browse/SPARK-25339), please comment here about your thoughts and

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21688 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96081/ Test PASSed. ---

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21688 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-09-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21688 **[Test build #96081 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96081/testReport)** for PR 21688 at commit

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22305 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22305 **[Test build #96084 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96084/testReport)** for PR 22305 at commit

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22305 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96084/ Test FAILed. ---

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22305 **[Test build #96084 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96084/testReport)** for PR 22305 at commit

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22305 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22305 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-14 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/22305 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22305 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22305 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96083/ Test FAILed. ---

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22305 **[Test build #96083 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96083/testReport)** for PR 22305 at commit

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22305 **[Test build #96083 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96083/testReport)** for PR 22305 at commit

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22305 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22305: [WIP][SPARK-24561][SQL][Python] User-defined window aggr...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22305 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #22355: [SPARK-25358][SQL] MutableProjection supports fal...

2018-09-14 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/22355#discussion_r217841164 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/InterpretedMutableProjection.scala --- @@ -0,0 +1,83 @@ +/* + *

[GitHub] spark pull request #22392: [SPARK-23200] Reset Kubernetes-specific config on...

2018-09-14 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22392#discussion_r217838764 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala --- @@ -54,6 +54,10 @@ class Checkpoint(ssc: StreamingContext, val

[GitHub] spark pull request #22416: [SPARK-25291][K8S][BACKPORT] Fixing Flakiness of ...

2018-09-14 Thread ifilonenko
Github user ifilonenko closed the pull request at: https://github.com/apache/spark/pull/22416 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21451: [SPARK-24296][CORE] Replicate large blocks as a stream.

2018-09-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21451 https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test/job/spark-branch-2.4-test-maven-hadoop-2.7/21/ --- - To

[GitHub] spark issue #21451: [SPARK-24296][CORE] Replicate large blocks as a stream.

2018-09-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21451 ``` org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in stage 0.0 (TID 7, localhost, executor 1):

[GitHub] spark issue #22420: SPARK-25429

2018-09-14 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22420 @hellodengfei . Could you make a PR against `master` branch since the target code is the same with `master` branch? We usually merge into `master` first, then start to backport to 2.4 and

[GitHub] spark pull request #22392: [SPARK-23200] Reset Kubernetes-specific config on...

2018-09-14 Thread ssaavedra
Github user ssaavedra commented on a diff in the pull request: https://github.com/apache/spark/pull/22392#discussion_r217821786 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala --- @@ -54,6 +54,10 @@ class Checkpoint(ssc: StreamingContext, val

[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...

2018-09-14 Thread cclauss
Github user cclauss commented on the issue: https://github.com/apache/spark/pull/22231 @holdenk Can you please do me a big favor and create a separate PR that makes sure that we are doing at least __flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics__ on

[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...

2018-09-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22231 **[Test build #96082 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96082/testReport)** for PR 22231 at commit

[GitHub] spark issue #22395: [SPARK-16323][SQL] Add IntegralDivide expression

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22395 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96079/ Test PASSed. ---

[GitHub] spark issue #22395: [SPARK-16323][SQL] Add IntegralDivide expression

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22395 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22395: [SPARK-16323][SQL] Add IntegralDivide expression

2018-09-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22395 **[Test build #96079 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96079/testReport)** for PR 22395 at commit

[GitHub] spark pull request #21654: [SPARK-24671][PySpark] DataFrame length using a d...

2018-09-14 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/21654#discussion_r217808692 --- Diff: python/pyspark/sql/dataframe.py --- @@ -375,6 +375,9 @@ def _truncate(self): return int(self.sql_ctx.getConf(

[GitHub] spark pull request #22396: [SPARK-23425][SQL][FOLLOWUP] Support wildcards in...

2018-09-14 Thread sujith71955
Github user sujith71955 commented on a diff in the pull request: https://github.com/apache/spark/pull/22396#discussion_r217802972 --- Diff: docs/sql-programming-guide.md --- @@ -1898,6 +1898,7 @@ working with timestamps in `pandas_udf`s to get the best performance, see -

[GitHub] spark pull request #21688: [SPARK-21809] : Change Stage Page to use datatabl...

2018-09-14 Thread pgandhi999
Github user pgandhi999 commented on a diff in the pull request: https://github.com/apache/spark/pull/21688#discussion_r217799929 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -962,7 +709,7 @@ private[ui] class TaskPagedTable( } }

[GitHub] spark issue #20556: [SPARK-23367][Build] Include python document style check...

2018-09-14 Thread rekhajoshm
Github user rekhajoshm commented on the issue: https://github.com/apache/spark/pull/20556 Yes @holdenk , seems like I missed the last notification on it. Please let me look into it this weekend. Thanks. --- - To

[GitHub] spark issue #22396: [SPARK-23425][SQL][FOLLOWUP] Support wildcards in HDFS p...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22396 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22396: [SPARK-23425][SQL][FOLLOWUP] Support wildcards in HDFS p...

2018-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22396 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96080/ Test PASSed. ---

[GitHub] spark issue #22396: [SPARK-23425][SQL][FOLLOWUP] Support wildcards in HDFS p...

2018-09-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22396 **[Test build #96080 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96080/testReport)** for PR 22396 at commit

[GitHub] spark pull request #21688: [SPARK-21809] : Change Stage Page to use datatabl...

2018-09-14 Thread pgandhi999
Github user pgandhi999 commented on a diff in the pull request: https://github.com/apache/spark/pull/21688#discussion_r217797298 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -185,89 +185,9 @@ private[ui] class StagePage(parent: StagesTab, store:

[GitHub] spark pull request #21688: [SPARK-21809] : Change Stage Page to use datatabl...

2018-09-14 Thread pgandhi999
Github user pgandhi999 commented on a diff in the pull request: https://github.com/apache/spark/pull/21688#discussion_r217797349 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/StagesResource.scala --- @@ -40,8 +43,25 @@ private[v1] class StagesResource extends

[GitHub] spark pull request #21688: [SPARK-21809] : Change Stage Page to use datatabl...

2018-09-14 Thread pgandhi999
Github user pgandhi999 commented on a diff in the pull request: https://github.com/apache/spark/pull/21688#discussion_r217797129 --- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js --- @@ -0,0 +1,807 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #21688: [SPARK-21809] : Change Stage Page to use datatabl...

2018-09-14 Thread pgandhi999
Github user pgandhi999 commented on a diff in the pull request: https://github.com/apache/spark/pull/21688#discussion_r217796781 --- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js --- @@ -0,0 +1,807 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #21688: [SPARK-21809] : Change Stage Page to use datatabl...

2018-09-14 Thread pgandhi999
Github user pgandhi999 commented on a diff in the pull request: https://github.com/apache/spark/pull/21688#discussion_r217796742 --- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js --- @@ -0,0 +1,807 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #21688: [SPARK-21809] : Change Stage Page to use datatabl...

2018-09-14 Thread pgandhi999
Github user pgandhi999 commented on a diff in the pull request: https://github.com/apache/spark/pull/21688#discussion_r217796714 --- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js --- @@ -0,0 +1,807 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #21688: [SPARK-21809] : Change Stage Page to use datatabl...

2018-09-14 Thread pgandhi999
Github user pgandhi999 commented on a diff in the pull request: https://github.com/apache/spark/pull/21688#discussion_r217796686 --- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js --- @@ -0,0 +1,807 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #21688: [SPARK-21809] : Change Stage Page to use datatabl...

2018-09-14 Thread pgandhi999
Github user pgandhi999 commented on a diff in the pull request: https://github.com/apache/spark/pull/21688#discussion_r217796571 --- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js --- @@ -0,0 +1,807 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #21688: [SPARK-21809] : Change Stage Page to use datatabl...

2018-09-14 Thread pgandhi999
Github user pgandhi999 commented on a diff in the pull request: https://github.com/apache/spark/pull/21688#discussion_r217796626 --- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js --- @@ -0,0 +1,807 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #21688: [SPARK-21809] : Change Stage Page to use datatabl...

2018-09-14 Thread pgandhi999
Github user pgandhi999 commented on a diff in the pull request: https://github.com/apache/spark/pull/21688#discussion_r217795714 --- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js --- @@ -0,0 +1,807 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #22421: [SPARK-25431][SQL][EXAMPLES] Fix function example...

2018-09-14 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22421#discussion_r217795552 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -718,7 +718,7 @@ case class

[GitHub] spark pull request #21688: [SPARK-21809] : Change Stage Page to use datatabl...

2018-09-14 Thread pgandhi999
Github user pgandhi999 commented on a diff in the pull request: https://github.com/apache/spark/pull/21688#discussion_r217795339 --- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js --- @@ -0,0 +1,807 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #21688: [SPARK-21809] : Change Stage Page to use datatabl...

2018-09-14 Thread pgandhi999
Github user pgandhi999 commented on a diff in the pull request: https://github.com/apache/spark/pull/21688#discussion_r217795298 --- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js --- @@ -0,0 +1,807 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-09-14 Thread raghavgautam
Github user raghavgautam commented on the issue: https://github.com/apache/spark/pull/22414 @maropu Can you continue with the review ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-09-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21688 **[Test build #96081 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96081/testReport)** for PR 21688 at commit

[GitHub] spark issue #22421: [SPARK-25431][SQL][EXAMPLES] Fix function examples and u...

2018-09-14 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22421 If you don't mind, please revert this. cc @cloud-fan --- - To unsubscribe, e-mail:

[GitHub] spark issue #22414: [SPARK-25424][SQL] Window duration and slide duration wi...

2018-09-14 Thread raghavgautam
Github user raghavgautam commented on the issue: https://github.com/apache/spark/pull/22414 The code below throws following error. Note that the error is produced at the time of count() call instead of window() call. ```java val df = spark.readStream .format("rate")

[GitHub] spark pull request #22414: [SPARK-25424][SQL] Window duration and slide dura...

2018-09-14 Thread raghavgautam
Github user raghavgautam commented on a diff in the pull request: https://github.com/apache/spark/pull/22414#discussion_r217793474 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/TimeWindowSuite.scala --- @@ -122,11 +122,57 @@ class TimeWindowSuite

[GitHub] spark issue #22421: [SPARK-25431][SQL][EXAMPLES] Fix function examples and u...

2018-09-14 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22421 @ueshin and @gatorsmile . Sorry, but could you explain where did you get the new result? IIRC, historically, the example illustrates the result from `spark-sql` like Hive

[GitHub] spark pull request #22414: [SPARK-25424][SQL] Window duration and slide dura...

2018-09-14 Thread raghavgautam
Github user raghavgautam commented on a diff in the pull request: https://github.com/apache/spark/pull/22414#discussion_r217793345 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/TimeWindowSuite.scala --- @@ -122,11 +122,57 @@ class TimeWindowSuite

[GitHub] spark pull request #22414: [SPARK-25424][SQL] Window duration and slide dura...

2018-09-14 Thread raghavgautam
Github user raghavgautam commented on a diff in the pull request: https://github.com/apache/spark/pull/22414#discussion_r217793214 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/TimeWindow.scala --- @@ -35,29 +35,15 @@ case class TimeWindow(

[GitHub] spark pull request #22414: [SPARK-25424][SQL] Window duration and slide dura...

2018-09-14 Thread raghavgautam
Github user raghavgautam commented on a diff in the pull request: https://github.com/apache/spark/pull/22414#discussion_r217793251 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/TimeWindowSuite.scala --- @@ -122,11 +122,57 @@ class TimeWindowSuite

  1   2   3   >