Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21804
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21804
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21804
**[Test build #93252 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93252/testReport)**
for PR 21804 at commit
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21802#discussion_r203560440
--- Diff: python/pyspark/sql/functions.py ---
@@ -2382,6 +2382,20 @@ def array_sort(col):
return
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20146
**[Test build #93246 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93246/testReport)**
for PR 20146 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/20146
Seems it is timeout: `Build execution time has reached the maximum allowed
time for your plan (90 minutes).` Can we set up the building time?
---
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/spark/pull/21700#discussion_r203577561
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/StateStoreSuite.scala
---
@@ -64,21 +66,143 @@ class StateStoreSuite
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21810
Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21700
**[Test build #93256 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93256/testReport)**
for PR 21700 at commit
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21777
> Except the TPC-DS queries, are we able to find some workloads that could
perform faster using the bytecode generated by the JDK compiler?
Since I don't have real workloads or non-TPCDS
GitHub user hehuiyuan opened a pull request:
https://github.com/apache/spark/pull/21812
SPARK UI K8S : this parameter's
illustration(spark.kubernetes.executor.label.[LabelName] )
[é¾æ¥å°åï¼](http://spark.apache.org/docs/2.3.0/running-on-kubernetes.html)
Github user echarles commented on the issue:
https://github.com/apache/spark/pull/21748
> Label spark-app-id is only set if spark-submit goes through the steps to
create the driver pod so doesn't apply in this case.
In that case, the client process could create its own
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/21794
`spark-sql` suggests that -0 and 0 are considered the same though. `SELECT
-0.0 == 0.0;` returns `true`. It's probably essential not to change behavior
here, but if performance is the issue, I think
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/21761
Hi, @gatorsmile .
Could you review this PR? I'm wondering if we can have this for Spark 2.4
before branch-cut.
---
-
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21806
@hvanhovell Got it. Thanks for your explanation. LGTM.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/21589
@MaxGekk We are going in circles.
I dont think this is a good api to expose currently - the data is available
through multiple other means as I detailed and while not a succinct oneliner,
it is
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21810
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93251/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21810
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21810
**[Test build #93251 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93251/testReport)**
for PR 21810 at commit
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21261
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/20177
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/17894
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/18034
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/20319
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/20543
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/19758
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/18918
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/13143
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21726
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/17422
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/19420
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/18125
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/19456
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/17092
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/17619
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/17973
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/18229
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/20304
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user echarles commented on the issue:
https://github.com/apache/spark/pull/21748
Got you points. About labels, right, we could take the road of the code
that creates labels on its own pod. To ensure uniqueness, we could use the
`spark-app-id` as key (if it maps the requirement
GitHub user MaxGekk opened a pull request:
https://github.com/apache/spark/pull/21810
[SPARK-24854][SQL] Gathering all Avro options into the AvroOptions class
## What changes were proposed in this pull request?
In the PR, I propose to put all `Avro` options in new class
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21810
**[Test build #93251 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93251/testReport)**
for PR 21810 at commit
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21589
> it's not terribly useful to know, e.g., that there are 5 million cores in
the cluster if your Job is running in a scheduler pool that is restricted to
using far fewer CPUs via the pool's
Github user ssimeonov commented on the issue:
https://github.com/apache/spark/pull/21589
@markhamstra I am confused about your API evaluation criteria.
You are not arguing about the specific benefits these changes can provide
immediately to an increasing majority of Spark
Github user markhamstra commented on the issue:
https://github.com/apache/spark/pull/21589
Thank you, @HyukjinKwon
There are a significant number of Spark users who use the Job Scheduler
model with a SparkContext shared across many users and many Jobs. Promoting
tools and
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21729
**[Test build #93249 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93249/testReport)**
for PR 21729 at commit
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
> Yes, the service gets its endpoints by matching its label selector
against labels on the pods so it's critical to have matching labels. Another
tenable solution is for the driver backend code to
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/21748
> That is why I suggested also to remove the driver's knowledge of the
driver pod name and to remove the owner reference concept entirely.
While, not worrying about the driver pod name
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21451
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21202
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21451
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21810
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user ssimeonov commented on the issue:
https://github.com/apache/spark/pull/21589
@markhamstra even the words you are using indicate that you are missing the
intended audience.
> high-level, declarative abstraction that can be used to specify requested
Job
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21488#discussion_r203561847
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetReader.scala
---
@@ -115,7 +116,7 @@ private[kafka010] class
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20838
**[Test build #93253 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93253/testReport)**
for PR 20838 at commit
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/spark/pull/21700#discussion_r203577783
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/HDFSBackedStateStoreProvider.scala
---
@@ -270,11 +273,43 @@
Github user ssimeonov commented on the issue:
https://github.com/apache/spark/pull/21589
> Repartitioning based upon a snapshot of the number of cores available
cluster-wide is clearly not the correct thing to do in many instances and use
cases.
I wholeheartedly agree and I
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21469
**[Test build #93257 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93257/testReport)**
for PR 21469 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20146
Meanwhile, will try to take another look to reduce the time, or see if we
can split the test, or we can request the time limit increase again as a last
resort.
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20146
Yea, that looks due to time limit. I am seeing sometimes it hit the time
limit issue.
The problem is, it's kind of difficult to increase the build time (see also
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/19510
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/18268
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/14653
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/19274
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/20437
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/20090
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/20100
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/16910
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21731
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21453
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21455
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21812
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21777
btw, it seems this pr exceeds the current timeout...ãAny way to
temporarily make the timeout longer? We always need to configure timeout in
the Jenkins-side like
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/18477
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21781
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21787
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21095
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/19233
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21240
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/12904
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/12951
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21635#discussion_r203594956
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMasterSource.scala
---
@@ -0,0 +1,49 @@
+/*
+ *
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21795
thanks, merging to master!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21589
> ... unless explicitly overridden by user.
This is the problem this PR addresses, actually.
> If you need fine grained information about executors, use spark listener
(it is
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/21748
> The problem is that the driver's labels might not be unique to that
driver, which therefore would require the user to assign their own unique
labels or for us to patch the driver pod in-place
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/21806
@viirya this current change is only useful when you compare canonicalized
plans created on different JVMs. This has come up when we tried to detect
changes in plans over spark versions (plan
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
I think taking a step back, it seems unwise more so to be making any
assumptions about the location in which a driver is running in client mode.
Client mode is simply just saying that the
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/21807
@NiharS yeah that makes sense. @mauropalsgraaf we missed this today (sorry
about that). Can you add the null check (bonus points if you call `eval()` only
once), add a test for this case?
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21808
**[Test build #93241 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93241/testReport)**
for PR 21808 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21808
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93241/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21803
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93242/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21803
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21803
**[Test build #93242 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93242/testReport)**
for PR 21803 at commit
Github user markhamstra commented on the issue:
https://github.com/apache/spark/pull/21589
No, defaultParallelism isn't more useful in that case, but that just starts
getting to my overall assessment of this JIRA and PR: It smells of defining the
problem to align with a preconception
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/21795
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user markhamstra commented on the issue:
https://github.com/apache/spark/pull/21589
It is precisely because the audience that I am concerned with is not
limited to just data scientists or notebook users and their particular needs
that I am far from convinced that exposing
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/21808
Merged to master
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
GitHub user countmdm opened a pull request:
https://github.com/apache/spark/pull/21811
[SPARK-24801][CORE] Avoid memory waste by empty byte[] arrays in
SaslEncryption$EncryptedMessage
## What changes were proposed in this pull request?
Initialize
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21805
Can you add tests? cc: @gatorsmile
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21451
**[Test build #93255 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93255/testReport)**
for PR 21451 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21729
**[Test build #93249 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93249/testReport)**
for PR 21729 at commit
1 - 100 of 585 matches
Mail list logo
