Github user maropu commented on the issue:
https://github.com/apache/spark/pull/18697
@aray Can you close this for now because it's not active for a long time?
(I'm not sure the current master still has this issue..., so you should check
it first)
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21848
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user sahilTakiar commented on the issue:
https://github.com/apache/spark/pull/21849
@vanzin could you take a look? I'm not sure if the same race conditions
present in the other unit tests apply to the new ones since no `SparkContext`
is being created. For now I didn't add any
Github user Achuth17 commented on the issue:
https://github.com/apache/spark/pull/21608
@gatorsmile @maropu ping.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21849
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21849
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21848
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #93455 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93455/testReport)**
for PR 21546 at commit
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21848#discussion_r204513622
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala
---
@@ -390,6 +390,7 @@ object SimplifyConditionals
Github user pgandhi999 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21809#discussion_r204513701
--- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala
---
@@ -94,6 +94,13 @@ private[spark] class AppStatusStore(
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/21811
I see. SGTM.
Would it be possible to attach heap profiling (allocated size for each
type) before and after this PR to record the difference?
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21839
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21839
**[Test build #93448 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93448/testReport)**
for PR 21839 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21839
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93448/
Test PASSed.
---
Github user countmdm commented on the issue:
https://github.com/apache/spark/pull/21811
Thank you very much for your responses, @squito. I agree with all you said.
@kiszk the heap dump that prompted me to make this change was obtained from
a customer, who probably ran into
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21848
**[Test build #93454 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93454/testReport)**
for PR 21848 at commit
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21746
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21839
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/21848#discussion_r204546087
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala
---
@@ -390,6 +390,7 @@ object SimplifyConditionals extends
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21837
**[Test build #93461 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93461/testReport)**
for PR 21837 at commit
Github user tedyu commented on the issue:
https://github.com/apache/spark/pull/21488
@zsxwing
Is there anything that needs to be done from my side ?
---
-
To unsubscribe, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21850
**[Test build #93462 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93462/testReport)**
for PR 21850 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21850
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21850
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21758
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93452/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21758
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21809
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21809
**[Test build #93458 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93458/testReport)**
for PR 21809 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20699
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93449/
Test PASSed.
---
Github user countmdm commented on the issue:
https://github.com/apache/spark/pull/21811
@kiszk the situation "before" is well understood. In the respective
SPARK-24801 ticket I present a fragment from the analysis of this heap dump by
jxray (www.jxray.com). It shows that ~2.5GB of
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93455/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21851
**[Test build #93459 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93459/testReport)**
for PR 21851 at commit
Github user huaxingao commented on the issue:
https://github.com/apache/spark/pull/21835
@HyukjinKwon @felixcheung
Could you please review? Thank you very much in advance!
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21809
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93458/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20699
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21848#discussion_r204524410
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala
---
@@ -390,6 +390,7 @@ object SimplifyConditionals
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21650
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/21852
This PR also has [the similar
issue](https://github.com/apache/spark/pull/21848#issuecomment-407166299) if a
condition has a side effect.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21650
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93450/
Test PASSed.
---
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21839
Thanks! Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
Merging in a few hours if no additional comments are raised.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user c-horn commented on the issue:
https://github.com/apache/spark/pull/21676
already resolved by https://github.com/apache/spark/pull/21746
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21488
@tedyu I just realized topic deletion might be used by some users. This
Kafka version will introduce a regression for them. Since 2.0 fixes this issue
and is going out soon, how about just wait for
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21650
**[Test build #93451 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93451/testReport)**
for PR 21650 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21848
For now, seems we don't have a good way to know if an expression has side
effect. Some expressions like `AssertTrue` should be marked as one with side
effect. Maybe we should create a trait for this
Github user c-horn closed the pull request at:
https://github.com/apache/spark/pull/21676
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user MaxGekk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21439#discussion_r204553995
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala
---
@@ -101,6 +102,17 @@ class JacksonParser(
}
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #93455 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93455/testReport)**
for PR 21546 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21852
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93460/
Test PASSed.
---
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21848
This is a good point.
On Mon, Jul 23, 2018, 12:03 PM Dongjoon Hyun
wrote:
> Since this skips the evaluation of if condition, this will cause the
> following difference.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21811
**[Test build #4221 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4221/testReport)**
for PR 21811 at commit
GitHub user dbtsai opened a pull request:
https://github.com/apache/spark/pull/21852
[SPARK-24893] [SQL] Remove the entire CaseWhen if all the outputs are
semantic equivalence
## What changes were proposed in this pull request?
Similar to SPARK-24890, if all the outputs of
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21650
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21650
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93451/
Test FAILed.
---
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21851#discussion_r204531052
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -2145,14 +2145,24 @@ class Analyzer(
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21848
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93454/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21848
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/21850#discussion_r204556169
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala
---
@@ -414,6 +414,12 @@ object SimplifyConditionals
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21758
**[Test build #93452 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93452/testReport)**
for PR 21758 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21853
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21851
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93459/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21851
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21852
**[Test build #93460 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93460/testReport)**
for PR 21852 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21852
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21852
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/21811
@countmdm I see. We may be interested in ratio regarding `byte[] / all
allocated memory` before and after. Not interested in other objects (e.g. an
object including customer's name).
---
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21837
> is there any or easy way to check the metadata for compression level?
@HyukjinKwon I am not sure the level exists in the metadata. At least
`avro-tools` doesn't show it.
```
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21850
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93456/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21850
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/16677#discussion_r204580788
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/TakeOrderedAndProjectSuite.scala
---
@@ -22,6 +22,7 @@ import scala.util.Random
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21852
**[Test build #93460 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93460/testReport)**
for PR 21852 at commit
Github user tedyu commented on the issue:
https://github.com/apache/spark/pull/21488
Ryan:
Thanks for the close follow-up.
Once Kafka 2.0.0 is released, I will incorporate the above.
---
-
To
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21852
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user dbtsai commented on the issue:
https://github.com/apache/spark/pull/21850
@cloud-fan and @gatorsmile
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21650
**[Test build #93450 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93450/testReport)**
for PR 21650 at commit
Github user dbtsai commented on the issue:
https://github.com/apache/spark/pull/21848
@gatorsmile this can remove some of the expensive condition expressions, so
I would like to find a way to properly implement this.
Thank you all for chiming in with many good points. Let me
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/21850#discussion_r204560250
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala
---
@@ -414,6 +414,12 @@ object SimplifyConditionals
GitHub user dilipbiswal opened a pull request:
https://github.com/apache/spark/pull/21853
[SPARK-23957][SQL] Sorts in subqueries are redundant and can be removed
## What changes were proposed in this pull request?
Thanks to @henryr for the original idea at
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21853
**[Test build #93463 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93463/testReport)**
for PR 21853 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21853
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21853
cc @maryannxue
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/21306
@cloud-fan, @gatorsmile, I don't think this should be merged yet.
I've been implementing CTAS and RTAS based on this commit and I don't think
it makes sense to get a `TableCatalog` instance
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21488
@tedyu fyi, here is the last fix for 2.0.0:
https://github.com/zsxwing/spark/commit/7ccc520b540106ca4c2dac2e42834984332d6804
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21850
**[Test build #93456 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93456/testReport)**
for PR 21850 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21764
Thanks! Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21847
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/21826
No we can't because you can still use string concat in filters, e.g.
colA || colB == "ab"
What is "||" here?
---
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/21840#discussion_r204476440
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -3858,3 +3858,29 @@ object ArrayUnion
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21805
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user c-horn commented on the issue:
https://github.com/apache/spark/pull/21752
Do I need to change anything else with this PR?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/21848#discussion_r204508986
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala
---
@@ -651,6 +652,7 @@ object
Github user pgandhi999 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21809#discussion_r204514110
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala ---
@@ -182,6 +184,12 @@ private[ui] class StagePage(parent: StagesTab, store:
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/21848#discussion_r204516658
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala
---
@@ -390,6 +390,7 @@ object SimplifyConditionals extends
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21851
**[Test build #93459 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93459/testReport)**
for PR 21851 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21851
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21851
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/18954
@tejasapatil Can you close this for now because it's not active for a long
time.
---
-
To unsubscribe, e-mail:
1 - 100 of 484 matches
Mail list logo
