Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22894
Merged to master
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22965
**[Test build #98553 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98553/testReport)**
for PR 22965 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22957
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22957
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22965
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22965
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22965#discussion_r231541613
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcReadBenchmark.scala ---
@@ -266,8 +268,9 @@ object OrcReadBenchmark extends
GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/22965
[SPARK-25964][SQL][Minor] Revise OrcReadBenchmark/DataSourceReadBenchmark
case names and execution instructions
## What changes were proposed in this pull request?
1.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22957
**[Test build #98552 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98552/testReport)**
for PR 22957 at commit
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/22955
cc @cloud-fan @mgaido91
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/22938
At least it doesn't fail on the cases
https://github.com/apache/spark/pull/22938/files#diff-6626026091295ad8c0dfb66ecbcd04b1R568
and
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/22938
I guess the problem belongs to `FailureSafeParser`, in particular
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/22962
cc @cloud-fan @gatorsmile
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22855#discussion_r231526283
--- Diff:
core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala ---
@@ -92,6 +94,20 @@ class KryoSerializer(conf: SparkConf)
new
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22855#discussion_r231526925
--- Diff:
core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala ---
@@ -214,8 +230,14 @@ class KryoSerializer(conf: SparkConf)
kryo
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22855
**[Test build #4418 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4418/testReport)**
for PR 22855 at commit
Github user koeninger commented on the issue:
https://github.com/apache/spark/pull/21038
@SehanRathnayake Kafka is designed for at most one consumer per partition
per consumer group at any given point in time,
https://kafka.apache.org/documentation/#design_consumerposition
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22921
**[Test build #98551 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98551/testReport)**
for PR 22921 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22921
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22921
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/22938
@HyukjinKwon Sure, the test would be for invalid JSON array:
```Scala
test("from_json - input=invalid JSON array, schema=array, output=array") {
val input = """[{"a": 1},
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22938
@attilapiros, mind showing rough small test codes for it please? just want
to see if this is something we should fix or not.
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22938
Yea, looks fine in general. Will take a look within this week or weekends.
---
-
To unsubscribe, e-mail:
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/22938
@MaxGekk I have checked out your PR and played a little bit with it:
created a new unit test as a copy of "from_json - input=array, schema=array,
output=array" with an invalid JSON.
I
Github user SehanRathnayake commented on the issue:
https://github.com/apache/spark/pull/21038
I think "This is fundamentally not correct, different apps should use
different consumer group" statement is wrong. .According to Kafka Having
consumers as part of the same consumer group
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22939
**[Test build #98550 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98550/testReport)**
for PR 22939 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22939
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98550/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22939
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22964
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22964
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22964
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user uzadude opened a pull request:
https://github.com/apache/spark/pull/22964
[SPARK-25963] Optimize generate followed by window
## What changes were proposed in this pull request?
I've added an optimizer rule to add Repartition operator when we have a
Generate operator
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19373
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22818
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user squito commented on the issue:
https://github.com/apache/spark/pull/22818
merged to master
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22951
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22951
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98543/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22951
**[Test build #98543 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98543/testReport)**
for PR 22951 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22939
**[Test build #98550 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98550/testReport)**
for PR 22939 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22939
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22939
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22939#discussion_r231476884
--- Diff: R/pkg/R/functions.R ---
@@ -205,11 +205,18 @@ NULL
#' also supported for the schema.
#' \item \code{from_csv}:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22963
cc @srowen, @holdenk and @rekhajoshm
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22963
**[Test build #98549 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98549/testReport)**
for PR 22963 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22963
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22963
**[Test build #98548 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98548/testReport)**
for PR 22963 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22963
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98548/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22963
**[Test build #98548 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98548/testReport)**
for PR 22963 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22963
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22963
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22963
**[Test build #98547 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98547/testReport)**
for PR 22963 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22963
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98547/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22963
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22963
**[Test build #98547 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98547/testReport)**
for PR 22963 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22963
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22963
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22963#discussion_r231463118
--- Diff: dev/lint-python ---
@@ -87,27 +91,46 @@ else
rm "$PYCODESTYLE_REPORT_PATH"
fi
-# stop the build if there are Python
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22963
**[Test build #98546 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98546/testReport)**
for PR 22963 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22963
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98546/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22963
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22963#discussion_r231462182
--- Diff: dev/lint-python ---
@@ -26,9 +26,13 @@
PYCODESTYLE_REPORT_PATH="$SPARK_ROOT_DIR/dev/pycodestyle-report.txt"
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22963
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22963
**[Test build #98546 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98546/testReport)**
for PR 22963 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22963
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22963#discussion_r231461879
--- Diff: dev/lint-python ---
@@ -26,9 +26,13 @@
PYCODESTYLE_REPORT_PATH="$SPARK_ROOT_DIR/dev/pycodestyle-report.txt"
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22963
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98545/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22963
**[Test build #98545 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98545/testReport)**
for PR 22963 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22963
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22963#discussion_r231460594
--- Diff: dev/lint-python ---
@@ -26,9 +26,13 @@
PYCODESTYLE_REPORT_PATH="$SPARK_ROOT_DIR/dev/pycodestyle-report.txt"
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22963
**[Test build #98545 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98545/testReport)**
for PR 22963 at commit
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22963#discussion_r231460041
--- Diff: dev/lint-python ---
@@ -87,27 +91,46 @@ else
rm "$PYCODESTYLE_REPORT_PATH"
fi
-# stop the build if there are Python
GitHub user HyukjinKwon opened a pull request:
https://github.com/apache/spark/pull/22963
[SPARK-25962][BUILD][PYTHON]Specify minimum versions for both pydocstyle
and flake8 in 'lint-python' script
## What changes were proposed in this pull request?
This PR explicitly
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/22598#discussion_r231458881
--- Diff:
core/src/main/scala/org/apache/spark/deploy/security/KafkaDelegationTokenProvider.scala
---
@@ -0,0 +1,65 @@
+/*
+ * Licensed to
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22962
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22962
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98544/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22962
**[Test build #98544 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98544/testReport)**
for PR 22962 at commit
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22305#discussion_r231408205
--- Diff: python/pyspark/sql/tests.py ---
@@ -6323,6 +6333,33 @@ def ordered_window(self):
def unpartitioned_window(self):
return
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22305#discussion_r231420974
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/WindowInPandasExec.scala
---
@@ -73,68 +118,147 @@ case class WindowInPandasExec(
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22305#discussion_r231427886
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/WindowInPandasExec.scala
---
@@ -73,68 +118,147 @@ case class WindowInPandasExec(
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22305#discussion_r231407486
--- Diff: python/pyspark/sql/tests.py ---
@@ -6323,6 +6333,33 @@ def ordered_window(self):
def unpartitioned_window(self):
return
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22305#discussion_r231429605
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/WindowInPandasExec.scala
---
@@ -73,68 +118,147 @@ case class WindowInPandasExec(
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22305#discussion_r231416643
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/WindowInPandasExec.scala
---
@@ -27,17 +27,62 @@ import
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22305#discussion_r231413246
--- Diff: python/pyspark/worker.py ---
@@ -154,6 +154,47 @@ def wrapped(*series):
return lambda *a: (wrapped(*a), arrow_return_type)
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22962
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22962
**[Test build #98544 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98544/testReport)**
for PR 22962 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22962
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
GitHub user xuanyuanking opened a pull request:
https://github.com/apache/spark/pull/22962
[SPARK-25921][PySpark] Fix BarrierTaskContext while python worker reuse
## What changes were proposed in this pull request?
While python worker reuse,
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/spark/pull/22952#discussion_r231429484
--- Diff: docs/structured-streaming-programming-guide.md ---
@@ -530,6 +530,8 @@ Here are the details of all the sources in Spark.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22954
For encryption stuff, I will try to handle that as well (maybe as a
followup(?)) so that we support it even when that's enabled.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22951
**[Test build #98543 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98543/testReport)**
for PR 22951 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22954
Thanks, @felixcheung. I will address those comments during cleaning up.
---
-
To unsubscribe, e-mail:
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22954#discussion_r231414561
--- Diff: R/pkg/R/SQLContext.R ---
@@ -147,6 +147,30 @@ getDefaultSqlSource <- function() {
l[["spark.sql.sources.default"]]
}
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22960#discussion_r231413853
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/CsvFunctionsSuite.scala ---
@@ -86,4 +86,82 @@ class CsvFunctionsSuite extends QueryTest with
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22951
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22951
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98541/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22960
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98542/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22960
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22951
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22960
**[Test build #98542 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98542/testReport)**
for PR 22960 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22951
**[Test build #98541 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98541/testReport)**
for PR 22951 at commit
301 - 400 of 400 matches
Mail list logo
