Cody Koeninger created SPARK-30570:
--
Summary: Update scalafmt to 1.0.3 with onlyChangedFiles feature
Key: SPARK-30570
URL: https://issues.apache.org/jira/browse/SPARK-30570
Project: Spark
Cody Koeninger created SPARK-26177:
--
Summary: Automated formatting for Scala code
Key: SPARK-26177
URL: https://issues.apache.org/jira/browse/SPARK-26177
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-26121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cody Koeninger resolved SPARK-26121.
Resolution: Fixed
Assignee: Anastasios Zouzias
Fix Version/s: 3.0.0
[
https://issues.apache.org/jira/browse/SPARK-25983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682464#comment-16682464
]
Cody Koeninger commented on SPARK-25983:
Looks like we need an equivalent warning in
[
https://issues.apache.org/jira/browse/SPARK-25983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682461#comment-16682461
]
Cody Koeninger commented on SPARK-25983:
Documentation already explicitly says not to link to
[
https://issues.apache.org/jira/browse/SPARK-25233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cody Koeninger resolved SPARK-25233.
Resolution: Fixed
Fix Version/s: 2.4.0
Issue resolved by pull request 3
[
https://issues.apache.org/jira/browse/SPARK-25233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cody Koeninger reassigned SPARK-25233:
--
Assignee: Reza Safi
> Give the user the option of specifying a fixed minimum message
[
https://issues.apache.org/jira/browse/SPARK-24987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569491#comment-16569491
]
Cody Koeninger commented on SPARK-24987:
It was merged to branch-2.3
[
https://issues.apache.org/jira/browse/SPARK-25026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569283#comment-16569283
]
Cody Koeninger commented on SPARK-25026:
I don't think distributions have ever included the
[
https://issues.apache.org/jira/browse/SPARK-24987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cody Koeninger resolved SPARK-24987.
Resolution: Fixed
Fix Version/s: 2.4.0
> Kafka Cached Consumer Leaking File
[
https://issues.apache.org/jira/browse/SPARK-24987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cody Koeninger reassigned SPARK-24987:
--
Assignee: Yuval Itzchakov
> Kafka Cached Consumer Leaking File Descriptors
>
[
https://issues.apache.org/jira/browse/SPARK-24713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cody Koeninger resolved SPARK-24713.
Resolution: Fixed
Fix Version/s: 2.4.0
Issue resolved by pull request 21690
[
https://issues.apache.org/jira/browse/SPARK-24713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cody Koeninger reassigned SPARK-24713:
--
Assignee: Yuanbo Liu
> AppMatser of spark streaming kafka OOM if there are hundreds
[
https://issues.apache.org/jira/browse/SPARK-19680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540138#comment-16540138
]
Cody Koeninger commented on SPARK-19680:
A new consumer group is the easiest thing to do, but if
[
https://issues.apache.org/jira/browse/SPARK-24720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16534797#comment-16534797
]
Cody Koeninger commented on SPARK-24720:
What's your plan to tell the difference between gaps
[
https://issues.apache.org/jira/browse/SPARK-24743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cody Koeninger resolved SPARK-24743.
Resolution: Fixed
Fix Version/s: 2.4.0
Issue resolved by pull request 21717
[
https://issues.apache.org/jira/browse/SPARK-18258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16525137#comment-16525137
]
Cody Koeninger commented on SPARK-18258:
[~Yohan123] This ticket is about giving implementors of
[
https://issues.apache.org/jira/browse/SPARK-24507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16518287#comment-16518287
]
Cody Koeninger commented on SPARK-24507:
You're welcome to submit a doc PR that clarifies that
[
https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16496633#comment-16496633
]
Cody Koeninger commented on SPARK-18057:
I'd just modify KafkaTestUtils to match the way things
[
https://issues.apache.org/jira/browse/SPARK-24067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474549#comment-16474549
]
Cody Koeninger commented on SPARK-24067:
[~zsxwing] even in situations where users weren't
[
https://issues.apache.org/jira/browse/SPARK-24067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cody Koeninger resolved SPARK-24067.
Resolution: Fixed
Fix Version/s: 2.3.1
Issue resolved by pull request 21300
[
https://issues.apache.org/jira/browse/SPARK-24067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452603#comment-16452603
]
Cody Koeninger commented on SPARK-24067:
The original PR
[
https://issues.apache.org/jira/browse/SPARK-24067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452337#comment-16452337
]
Cody Koeninger commented on SPARK-24067:
Given the response on the dev list about criteria for
[
https://issues.apache.org/jira/browse/SPARK-21168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cody Koeninger resolved SPARK-21168.
Resolution: Fixed
Fix Version/s: 2.4.0
Issue resolved by pull request 19887
[
https://issues.apache.org/jira/browse/SPARK-21168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cody Koeninger reassigned SPARK-21168:
--
Assignee: liuzhaokun
> KafkaRDD should always set kafka clientId.
>
[
https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16442838#comment-16442838
]
Cody Koeninger commented on SPARK-18057:
[~cricket007] here's a branch with spark 2.1.1 / kafka
[
https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441818#comment-16441818
]
Cody Koeninger commented on SPARK-18057:
Ok, if you can figure out what version of spark it is I
[
https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441793#comment-16441793
]
Cody Koeninger commented on SPARK-18057:
Just adding the extra dependency on 0.11 probably won't
[
https://issues.apache.org/jira/browse/SPARK-22968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cody Koeninger resolved SPARK-22968.
Resolution: Fixed
Fix Version/s: 2.4.0
Issue resolved by pull request 21038
[
https://issues.apache.org/jira/browse/SPARK-22968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cody Koeninger reassigned SPARK-22968:
--
Assignee: Saisai Shao
> java.lang.IllegalStateException: No current assignment for
[
https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441606#comment-16441606
]
Cody Koeninger commented on SPARK-18057:
Out of curiosity, was that a compacted topic?
>
[
https://issues.apache.org/jira/browse/SPARK-19680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433039#comment-16433039
]
Cody Koeninger commented on SPARK-19680:
[~nerdynick] If you submit a PR to add documentation
[
https://issues.apache.org/jira/browse/SPARK-23739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414326#comment-16414326
]
Cody Koeninger commented on SPARK-23739:
Ok, the OutOfMemoryError is probably a separate and
[
https://issues.apache.org/jira/browse/SPARK-23739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411503#comment-16411503
]
Cody Koeninger commented on SPARK-23739:
I meant the version of the org.apache.kafka
[
https://issues.apache.org/jira/browse/SPARK-23739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411396#comment-16411396
]
Cody Koeninger commented on SPARK-23739:
What version of the org.apache.kafka artifact is in the
[
https://issues.apache.org/jira/browse/SPARK-18580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cody Koeninger resolved SPARK-18580.
Resolution: Fixed
Target Version/s: 2.4.0
> Use
[
https://issues.apache.org/jira/browse/SPARK-18580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cody Koeninger reassigned SPARK-18580:
--
Assignee: Oleksandr Konopko
> Use spark.streaming.backpressure.initialRate in
[
https://issues.apache.org/jira/browse/SPARK-18371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cody Koeninger resolved SPARK-18371.
Resolution: Fixed
Fix Version/s: 2.4.0
> Spark Streaming backpressure bug -
[
https://issues.apache.org/jira/browse/SPARK-18371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cody Koeninger reassigned SPARK-18371:
--
Assignee: Sebastian Arzt
> Spark Streaming backpressure bug - generates a batch with
[
https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387245#comment-16387245
]
Cody Koeninger commented on SPARK-18057:
It's probably easiest to keep the KIP discussion on the
[
https://issues.apache.org/jira/browse/SPARK-19767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387217#comment-16387217
]
Cody Koeninger commented on SPARK-19767:
[~nafshartous] I think at this point people are more
[
https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370924#comment-16370924
]
Cody Koeninger commented on SPARK-18057:
Just doing the upgrade is probably a good starting point
[
https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370460#comment-16370460
]
Cody Koeninger commented on SPARK-18057:
My guess is that DStream based integrations aren't
[
https://issues.apache.org/jira/browse/SPARK-22561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cody Koeninger resolved SPARK-22561.
Resolution: Not A Problem
> Dynamically update topics list for spark kafka consumer
>
[
https://issues.apache.org/jira/browse/SPARK-22561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264643#comment-16264643
]
Cody Koeninger commented on SPARK-22561:
See SubscribePattern
[
https://issues.apache.org/jira/browse/SPARK-22486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248891#comment-16248891
]
Cody Koeninger commented on SPARK-22486:
Can you identify a clear use case for this, given that
[
https://issues.apache.org/jira/browse/SPARK-19680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16242438#comment-16242438
]
Cody Koeninger commented on SPARK-19680:
If you got an OffsetOutOfRangeException after a job had
[
https://issues.apache.org/jira/browse/SPARK-20928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16221025#comment-16221025
]
Cody Koeninger commented on SPARK-20928:
No, it doesn't exist yet as far as I know.
Reason I ask
[
https://issues.apache.org/jira/browse/SPARK-20928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16220694#comment-16220694
]
Cody Koeninger commented on SPARK-20928:
Can you clarify how this impacts sinks having access to
[
https://issues.apache.org/jira/browse/SPARK-20928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16202923#comment-16202923
]
Cody Koeninger commented on SPARK-20928:
If a given sink is handling a result, why does handling
[
https://issues.apache.org/jira/browse/SPARK-20928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16202905#comment-16202905
]
Cody Koeninger commented on SPARK-20928:
I was talking about the specific case of jobs with only
[
https://issues.apache.org/jira/browse/SPARK-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157002#comment-16157002
]
Cody Koeninger commented on SPARK-17147:
Patch is there, if anyone wants to test it and provide
[
https://issues.apache.org/jira/browse/SPARK-19680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076646#comment-16076646
]
Cody Koeninger commented on SPARK-19680:
Direct Stream can take a mapping from topicpartition to
[
https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068980#comment-16068980
]
Cody Koeninger commented on SPARK-18057:
Kafka 0.11 is now released.
Are we upgrading spark
[
https://issues.apache.org/jira/browse/SPARK-21233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16066517#comment-16066517
]
Cody Koeninger commented on SPARK-21233:
You already have the choice of where you want to store
[
https://issues.apache.org/jira/browse/SPARK-20928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16054690#comment-16054690
]
Cody Koeninger commented on SPARK-20928:
Cool, can you label it SPIP so it shows up linked from
[
https://issues.apache.org/jira/browse/SPARK-20928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16052831#comment-16052831
]
Cody Koeninger commented on SPARK-20928:
This needs an improvement proposal.
Based on
[
https://issues.apache.org/jira/browse/SPARK-20928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16035964#comment-16035964
]
Cody Koeninger commented on SPARK-20928:
For jobs that only have narrow stages, I think it should
[
https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15977709#comment-15977709
]
Cody Koeninger commented on SPARK-18057:
People have also been reporting that explicit dependency
[
https://issues.apache.org/jira/browse/SPARK-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973554#comment-15973554
]
Cody Koeninger commented on SPARK-20036:
[~danielnuriyev] what actually happened when you removed
[
https://issues.apache.org/jira/browse/SPARK-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973507#comment-15973507
]
Cody Koeninger commented on SPARK-20036:
I'll submit a PR to add a note to the docs about this.
[
https://issues.apache.org/jira/browse/SPARK-20287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cody Koeninger updated SPARK-20287:
---
What you're describing is closer to the receiver-based implementation,
which had a number of
[
https://issues.apache.org/jira/browse/SPARK-19976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966208#comment-15966208
]
Cody Koeninger commented on SPARK-19976:
What would your expected behavior be when you delete
[
https://issues.apache.org/jira/browse/SPARK-20037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966199#comment-15966199
]
Cody Koeninger commented on SPARK-20037:
I'd be inclined to say this is a duplicate of the issue
[
https://issues.apache.org/jira/browse/SPARK-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966193#comment-15966193
]
Cody Koeninger commented on SPARK-20036:
fixKafkaParams is related to executor consumers, not the
[
https://issues.apache.org/jira/browse/SPARK-20287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966180#comment-15966180
]
Cody Koeninger commented on SPARK-20287:
The issue here is that the underlying new Kafka consumer
[
https://issues.apache.org/jira/browse/SPARK-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943827#comment-15943827
]
Cody Koeninger commented on SPARK-19904:
It has been added to apache/spark-website git repo
[
https://issues.apache.org/jira/browse/SPARK-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15906593#comment-15906593
]
Cody Koeninger commented on SPARK-19904:
Up now at
[
https://issues.apache.org/jira/browse/SPARK-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cody Koeninger reassigned SPARK-19904:
--
Assignee: Cody Koeninger
> SPIP Add Spark Project Improvement Proposal doc to website
[
https://issues.apache.org/jira/browse/SPARK-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cody Koeninger updated SPARK-19904:
---
Issue Type: Improvement (was: Bug)
> SPIP Add Spark Project Improvement Proposal doc to
[
https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905774#comment-15905774
]
Cody Koeninger commented on SPARK-18057:
Based on previous kafka client upgrades I wouldn't
[
https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905747#comment-15905747
]
Cody Koeninger commented on SPARK-18057:
I think the bigger question is once there's a kafka
[
https://issues.apache.org/jira/browse/SPARK-19888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905730#comment-15905730
]
Cody Koeninger commented on SPARK-19888:
That stacktrace also shows a concurrent modification
[
https://issues.apache.org/jira/browse/SPARK-19863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905603#comment-15905603
]
Cody Koeninger commented on SPARK-19863:
Isn't this basically a duplicate of
SPARK-19185
with the
[
https://issues.apache.org/jira/browse/SPARK-19904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cody Koeninger updated SPARK-19904:
---
Description:
see
Cody Koeninger created SPARK-19904:
--
Summary: SPIP Add Spark Project Improvement Proposal doc to website
Key: SPARK-19904
URL: https://issues.apache.org/jira/browse/SPARK-19904
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15887091#comment-15887091
]
Cody Koeninger commented on SPARK-17147:
Dean if you guys have any bandwith to help test out
[
https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cody Koeninger updated SPARK-18057:
---
I think you should ask Michael and / or Ryan what their plan is.
> Update structured
[
https://issues.apache.org/jira/browse/SPARK-19680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878623#comment-15878623
]
Cody Koeninger edited comment on SPARK-19680 at 2/22/17 4:25 PM:
-
The
[
https://issues.apache.org/jira/browse/SPARK-19680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878623#comment-15878623
]
Cody Koeninger commented on SPARK-19680:
The issue here is likely that you have lost data
[
https://issues.apache.org/jira/browse/SPARK-19361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cody Koeninger resolved SPARK-19361.
Resolution: Duplicate
> kafka.maxRatePerPartition for compacted topic cause exception
>
[
https://issues.apache.org/jira/browse/SPARK-19361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15839683#comment-15839683
]
Cody Koeninger commented on SPARK-19361:
Compacted topics in general don't work with direct
[
https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837844#comment-15837844
]
Cody Koeninger commented on SPARK-18057:
If you can get commiter agreement on the outstanding
[
https://issues.apache.org/jira/browse/SPARK-19185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826527#comment-15826527
]
Cody Koeninger commented on SPARK-19185:
I'd expect setting cache capacity to zero to cause
[
https://issues.apache.org/jira/browse/SPARK-19185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15823164#comment-15823164
]
Cody Koeninger commented on SPARK-19185:
This is a good error report, sorry it's taken me a while
[
https://issues.apache.org/jira/browse/SPARK-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15742285#comment-15742285
]
Cody Koeninger commented on SPARK-17147:
If compacted topics are important to you, then you
[
https://issues.apache.org/jira/browse/SPARK-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15729055#comment-15729055
]
Cody Koeninger commented on SPARK-17147:
This ticket is about createDirectStream. The question
[
https://issues.apache.org/jira/browse/SPARK-18682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15720146#comment-15720146
]
Cody Koeninger commented on SPARK-18682:
Isn't this a duplicate of
[
https://issues.apache.org/jira/browse/SPARK-18682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cody Koeninger updated SPARK-18682:
---
Comment: was deleted
(was: Isn't this a duplicate of
[
https://issues.apache.org/jira/browse/SPARK-18682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15720145#comment-15720145
]
Cody Koeninger commented on SPARK-18682:
Isn't this a duplicate of
[
https://issues.apache.org/jira/browse/SPARK-18506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15712457#comment-15712457
]
Cody Koeninger commented on SPARK-18506:
Yes, amazon linux. No, not spark-ec2, just a spark
[
https://issues.apache.org/jira/browse/SPARK-18475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15706814#comment-15706814
]
Cody Koeninger commented on SPARK-18475:
Glad you agree it shouldn't be enabled by default.
If
[
https://issues.apache.org/jira/browse/SPARK-18475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15706758#comment-15706758
]
Cody Koeninger commented on SPARK-18475:
Burak hasn't empirically shown that it is of benefit for
[
https://issues.apache.org/jira/browse/SPARK-18506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704372#comment-15704372
]
Cody Koeninger commented on SPARK-18506:
1 x spark master is m3 medium
2 x spark workers are m3
[
https://issues.apache.org/jira/browse/SPARK-18506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15691386#comment-15691386
]
Cody Koeninger commented on SPARK-18506:
I'm really confused by that - did you try a completely
[
https://issues.apache.org/jira/browse/SPARK-18525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15690915#comment-15690915
]
Cody Koeninger commented on SPARK-18525:
Easiest thing to do is just restart your streaming job
[
https://issues.apache.org/jira/browse/SPARK-18525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15686746#comment-15686746
]
Cody Koeninger commented on SPARK-18525:
0.8 works only against defined partitions. Use 0.10 and
[
https://issues.apache.org/jira/browse/SPARK-18506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15685190#comment-15685190
]
Cody Koeninger commented on SPARK-18506:
I'd try to isolate aws vs gce as a possible cause before
[
https://issues.apache.org/jira/browse/SPARK-18506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682216#comment-15682216
]
Cody Koeninger commented on SPARK-18506:
I tried your example code on an AWS 2-node spark
[
https://issues.apache.org/jira/browse/SPARK-18475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682171#comment-15682171
]
Cody Koeninger commented on SPARK-18475:
An iterator certainly does have an ordering guarantee,
1 - 100 of 409 matches
Mail list logo