Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22533
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22530
**[Test build #96496 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96496/testReport)**
for PR 22530 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/22524
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22326
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22524
**[Test build #96501 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96501/testReport)**
for PR 22524 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22326
**[Test build #96503 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96503/testReport)**
for PR 22326 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22326
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22524
**[Test build #96499 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96499/testReport)**
for PR 22524 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22484
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22524
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96499/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22326
**[Test build #96498 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96498/testReport)**
for PR 22326 at commit
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/21747
If nobody has any objections, I'm happy to close this PR.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
GitHub user MaxGekk opened a pull request:
https://github.com/apache/spark/pull/22534
[SPARK-25514][SQL] Pretty JSON
## What changes were proposed in this pull request?
The PR introduces new JSON option `pretty` which allows to turn on
`DefaultPrettyPrinter` of `Jackson`'s
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22484
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22484
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22534
**[Test build #96504 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96504/testReport)**
for PR 22534 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22524
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22484
**[Test build #96502 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96502/testReport)**
for PR 22484 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22484
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22484
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22524
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22530
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96496/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22530
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22484
**[Test build #96500 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96500/testReport)**
for PR 22484 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22511
**[Test build #96497 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96497/testReport)**
for PR 22511 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22524
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22511
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96497/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22326
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96498/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22326
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22511
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22484
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96500/
Test FAILed.
---
GitHub user dbtsai opened a pull request:
https://github.com/apache/spark/pull/22535
[SPARK-17636][SQL][WIP] Parquet predicate pushdown in nested fields
## What changes were proposed in this pull request?
Support Parquet predicate pushdown in nested fields
## How
Github user dilipbiswal commented on the issue:
https://github.com/apache/spark/pull/22326
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user dilipbiswal commented on the issue:
https://github.com/apache/spark/pull/22484
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22525
Hi @dongjoon-hyun , seems Jenkins is down. could you please ask for test
again?
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22533
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user annamolchanova opened a pull request:
https://github.com/apache/spark/pull/22533
[SPARK-18818][PYTHON] Add 'ascending' parameter to Window.orderBy()
## What changes were proposed in this pull request?
Pass `ascending` parameter to Window's orderBy() function the
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22533
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22534
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22534
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user mareksimunek commented on a diff in the pull request:
https://github.com/apache/spark/pull/22485#discussion_r219786477
--- Diff:
common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java
---
@@ -168,6 +170,15 @@ protected void
Github user mareksimunek commented on a diff in the pull request:
https://github.com/apache/spark/pull/22485#discussion_r219786856
--- Diff:
resource-managers/yarn/src/test/scala/org/apache/spark/network/yarn/YarnShuffleServiceMetricsSuite.scala
---
@@ -0,0 +1,75 @@
+/*
+
Github user mareksimunek commented on the issue:
https://github.com/apache/spark/pull/22485
@tgravescs thanks for review, I made corrections for your notes and
probably need some permissions to retest. Could you please test this PR again?
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22536
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22326
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96503/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22326
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user tgravescs commented on the issue:
https://github.com/apache/spark/pull/22485
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/21524
@tengpeng there's no deadline, but 2.4 is being released imminently and
doc/example improvements can go in this week, probably. If it's going to be a
while before you can update, we generally close
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22526#discussion_r219827419
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala ---
@@ -685,7 +685,15 @@ private[ui] class TaskDataSource(
private var
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19868
Can we also update the title?
```
Avoid iterating all partition paths when
spark.sql.hive.verifyPartitionPath=true
```
This is not true, we didn't fix the problem of
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22379#discussion_r219814329
--- Diff: sql/core/src/test/resources/sql-tests/inputs/csv-functions.sql ---
@@ -0,0 +1,12 @@
+-- from_csv
+describe function from_csv;
Github user mgaido91 closed the pull request at:
https://github.com/apache/spark/pull/21184
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22529
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user skonto commented on the issue:
https://github.com/apache/spark/pull/22381
@vanzin AFAIK the operations just update metrics in the underlying
dropwizard metrics library. I dont htink anything is shipped anywhere. How
should I proceed? Is there anyone who is familiar with
Github user mareksimunek commented on a diff in the pull request:
https://github.com/apache/spark/pull/22485#discussion_r219789235
--- Diff:
common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java
---
@@ -168,6 +170,15 @@ protected void
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22524
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96501/
Test PASSed.
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22534
I'm supportive of this idea.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22534#discussion_r219809938
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JSONOptions.scala
---
@@ -113,6 +113,11 @@ private[sql] class JSONOptions(
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22534
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96504/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22534
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user mareksimunek commented on the issue:
https://github.com/apache/spark/pull/22485
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22379#discussion_r219813519
--- Diff: python/pyspark/sql/functions.py ---
@@ -2637,6 +2637,29 @@ def sequence(start, stop, step=None):
_to_java_column(start),
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22534
Oh I rushed to read. Shall we then document? Let's add simple set of end to
end tests for then as well.
---
-
To
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22536
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user abhisrao opened a pull request:
https://github.com/apache/spark/pull/22536
[SPARK-25500][K8s]Specify configmap and secrets in Spark driver and
executor pods in Kubernetes
## What changes were proposed in this pull request?
This PR introduces support for specifying
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22536
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22484
LGTM, cc @dongjoon-hyun for sign-off
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user mareksimunek commented on a diff in the pull request:
https://github.com/apache/spark/pull/22485#discussion_r219786991
--- Diff:
resource-managers/yarn/src/test/scala/org/apache/spark/network/yarn/YarnShuffleServiceMetricsSuite.scala
---
@@ -0,0 +1,75 @@
+/*
+
Github user mareksimunek commented on a diff in the pull request:
https://github.com/apache/spark/pull/22485#discussion_r219786943
--- Diff:
resource-managers/yarn/src/test/scala/org/apache/spark/network/yarn/YarnShuffleServiceMetricsSuite.scala
---
@@ -0,0 +1,75 @@
+/*
+
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22524
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22524
**[Test build #96501 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96501/testReport)**
for PR 22524 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22484
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96502/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22484
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22534
Let's clarify this options is only for json functions in PR title
---
-
To unsubscribe, e-mail:
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22533#discussion_r219811169
--- Diff: python/pyspark/sql/window.py ---
@@ -76,12 +79,37 @@ def partitionBy(*cols):
@staticmethod
@since(1.4)
-def
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22533#discussion_r219812531
--- Diff: python/pyspark/sql/window.py ---
@@ -76,12 +79,37 @@ def partitionBy(*cols):
@staticmethod
@since(1.4)
-def
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22379#discussion_r219814691
--- Diff: sql/core/src/test/resources/sql-tests/inputs/csv-functions.sql ---
@@ -0,0 +1,12 @@
+-- from_csv
+describe function from_csv;
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/22534
> Let's clarify this options is only for json functions in PR title
In general, the option can be used to produce pretty printed files with
JSON, for example in the multi-line mode. I
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22534
**[Test build #96504 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96504/testReport)**
for PR 22534 at commit
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22453#discussion_r219827092
--- Diff: docs/sql-programming-guide.md ---
@@ -1002,6 +1002,21 @@ Configuration of Parquet can be done using the
`setConf` method on `SparkSession
Github user mareksimunek commented on a diff in the pull request:
https://github.com/apache/spark/pull/22485#discussion_r219786966
--- Diff:
resource-managers/yarn/src/test/scala/org/apache/spark/network/yarn/YarnShuffleServiceMetricsSuite.scala
---
@@ -0,0 +1,75 @@
+/*
+
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22484
**[Test build #96502 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96502/testReport)**
for PR 22484 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22326
**[Test build #96503 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96503/testReport)**
for PR 22326 at commit
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22379#discussion_r219815230
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/CsvFunctionsSuite.scala ---
@@ -0,0 +1,74 @@
+/*
+ * Licensed to the Apache Software
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22326
Some thoughts:
1. This rule is a little tricky as it only handles python udf accessing
attributes from both side. If it only accesses one side, we assume it can be
pushed down later.
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22316
LGTM if the decimal precision concern from @HyukjinKwon is addressed.
---
-
To unsubscribe, e-mail:
Github user tgravescs commented on the issue:
https://github.com/apache/spark/pull/22485
test this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/22419
https://github.com/apache/spark/pull/22419#issuecomment-423454727
This approach looks good to me cuz it makes the implementation simpler.
But, there is one thing I worry about; truncating is a
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22526
**[Test build #4349 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4349/testReport)**
for PR 22526 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22494
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22466
There is ... see
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-ManagedandExternalTables
I think Spark conflates the two. It's rare (?) but possible to
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22526#discussion_r219876198
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala ---
@@ -685,7 +685,10 @@ private[ui] class TaskDataSource(
private
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22529
Merged to `branch-2.4`.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/22364
I also run on the TPCDS and TPCH benchmark with 10 runs:
Rule | Effective After | Effective Before | Total After | Total Before | %
Eff | % Total
-- | -- | -- | -- | -- | -- | --
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/18544
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22466
We should look at Spark documentation, and Hive, if any, to figure out what
the right behavior is here. Spark generally follows Hive. See
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22518
> This can happen for instance when a filter containing a scalar subquery
is pushed to a DataSource
hmm, how can this happen? I don't think a data source can handle a filter
of
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22526#discussion_r219869516
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala ---
@@ -685,7 +685,10 @@ private[ui] class TaskDataSource(
private var
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22325
Jenkins, retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/22485#discussion_r219860309
--- Diff:
common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleServiceMetrics.java
---
@@ -0,0 +1,133 @@
+/*
+ * Licensed
1 - 100 of 468 matches
Mail list logo
