[
https://issues.apache.org/jira/browse/SPARK-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14117825#comment-14117825
]
Sandy Ryza commented on SPARK-3179:
---
Hi Michael,
Happy to help review your code or answ
[
https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza updated SPARK-1239:
--
Summary: Don't fetch all map output statuses at each reducer during
shuffles (was: Don't fetch all map
[
https://issues.apache.org/jira/browse/SPARK-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14118286#comment-14118286
]
Sandy Ryza commented on SPARK-2978:
---
IIUC, that would require using ShuffledRDD directly
[
https://issues.apache.org/jira/browse/SPARK-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14119080#comment-14119080
]
Sandy Ryza commented on SPARK-2978:
---
What's the thinking behind adding sortWithinPartiti
[
https://issues.apache.org/jira/browse/SPARK-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14119091#comment-14119091
]
Sandy Ryza commented on SPARK-2978:
---
Ah ok, sounds good.
> Provide an MR-style shuffle
Sandy Ryza created SPARK-3360:
-
Summary: Add RowMatrix.multiply(Vector)
Key: SPARK-3360
URL: https://issues.apache.org/jira/browse/SPARK-3360
Project: Spark
Issue Type: Improvement
Comp
[
https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123238#comment-14123238
]
Sandy Ryza commented on SPARK-3174:
---
I've been putting a little bit of thought into this
Sandy Ryza created SPARK-3419:
-
Summary: Scheduler shouldn't delay running a task when executors
don't reside at any of its preferred locations
Key: SPARK-3419
URL: https://issues.apache.org/jira/browse/SPARK-3419
[
https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza updated SPARK-3174:
--
Attachment: SPARK-3174design.pdf
> Under YARN, add and remove executors based on load
>
[
https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123504#comment-14123504
]
Sandy Ryza commented on SPARK-3174:
---
Posted a high-level design doc.
> Under YARN, add
Sandy Ryza created SPARK-3422:
-
Summary: JavaAPISuite.getHadoopInputSplits isn't used anywhere
Key: SPARK-3422
URL: https://issues.apache.org/jira/browse/SPARK-3422
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123587#comment-14123587
]
Sandy Ryza commented on SPARK-2099:
---
Yeah, unfortunately I haven't had the chance to add
[
https://issues.apache.org/jira/browse/SPARK-3082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza resolved SPARK-3082.
---
Resolution: Fixed
Fix Version/s: 1.1.0
> yarn.Client.logClusterResourceDetails throws NPE if re
[
https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125235#comment-14125235
]
Sandy Ryza commented on SPARK-3174:
---
To be clear, by YARN shuffle you mean the MR2 appro
[
https://issues.apache.org/jira/browse/SPARK-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125936#comment-14125936
]
Sandy Ryza edited comment on SPARK-3441 at 9/8/14 7:09 PM:
---
Beca
[
https://issues.apache.org/jira/browse/SPARK-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125936#comment-14125936
]
Sandy Ryza commented on SPARK-3441:
---
I'll add mention that this can be used to get Hadoo
[
https://issues.apache.org/jira/browse/SPARK-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126192#comment-14126192
]
Sandy Ryza commented on SPARK-3441:
---
bq. One case where you may not care about giving a
[
https://issues.apache.org/jira/browse/SPARK-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126454#comment-14126454
]
Sandy Ryza commented on SPARK-3441:
---
Right. It's not much work, but there are some ques
[
https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14127415#comment-14127415
]
Sandy Ryza commented on SPARK-3174:
---
bq. Since you mention the graceful decommission as
Sandy Ryza created SPARK-3464:
-
Summary: Graceful decommission of executors
Key: SPARK-3464
URL: https://issues.apache.org/jira/browse/SPARK-3464
Project: Spark
Issue Type: Sub-task
R
[
https://issues.apache.org/jira/browse/SPARK-3460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza updated SPARK-3460:
--
Summary: Discard executors (was: Graceful decommission of idle YARN
sessions)
> Discard executors
> --
[
https://issues.apache.org/jira/browse/SPARK-3460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza updated SPARK-3460:
--
Summary: Under YARN, discard executors that have been idle (was: Discard
executors)
> Under YARN, dis
[
https://issues.apache.org/jira/browse/SPARK-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza updated SPARK-3464:
--
Description: In most cases, even when an application is utilizing only a
small fraction of its available
[
https://issues.apache.org/jira/browse/SPARK-3172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130475#comment-14130475
]
Sandy Ryza commented on SPARK-3172:
---
I mean in the web UI (which will require distinguis
Sandy Ryza created SPARK-3497:
-
Summary: Report serialized size of task binary
Key: SPARK-3497
URL: https://issues.apache.org/jira/browse/SPARK-3497
Project: Spark
Issue Type: Improvement
Sandy Ryza created SPARK-3560:
-
Summary: In yarn-cluster mode, jars are distributed through
multiple mechanisms.
Key: SPARK-3560
URL: https://issues.apache.org/jira/browse/SPARK-3560
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza updated SPARK-3560:
--
Component/s: YARN
> In yarn-cluster mode, jars are distributed through multiple mechanisms.
> --
[
https://issues.apache.org/jira/browse/SPARK-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14136882#comment-14136882
]
Sandy Ryza commented on SPARK-3560:
---
Right. I believe Min from LinkedIn who discovered
[
https://issues.apache.org/jira/browse/SPARK-3574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14138246#comment-14138246
]
Sandy Ryza commented on SPARK-3574:
---
On it
> Shuffle finish time always reported as -1
[
https://issues.apache.org/jira/browse/SPARK-3577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14138245#comment-14138245
]
Sandy Ryza commented on SPARK-3577:
---
On it
> Shuffle write time incorrect for sort-base
[
https://issues.apache.org/jira/browse/SPARK-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14138319#comment-14138319
]
Sandy Ryza commented on SPARK-3530:
---
bq. Isn't the "fit multiple models at once" part a
[
https://issues.apache.org/jira/browse/SPARK-3577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14138334#comment-14138334
]
Sandy Ryza commented on SPARK-3577:
---
Have you noticed the incorrect metrics reported or
[
https://issues.apache.org/jira/browse/SPARK-3577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14138354#comment-14138354
]
Sandy Ryza commented on SPARK-3577:
---
In the old code, the ShuffleWriteMetrics didn't get
[
https://issues.apache.org/jira/browse/SPARK-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza updated SPARK-3560:
--
Summary: In yarn-cluster mode, the same jars are distributed through
multiple mechanisms. (was: In yarn
[
https://issues.apache.org/jira/browse/SPARK-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14139958#comment-14139958
]
Sandy Ryza commented on SPARK-3573:
---
Currently SchemaRDD lives inside SQL. Would we mov
Sandy Ryza created SPARK-3605:
-
Summary: Typo in SchemaRDD JavaDoc
Key: SPARK-3605
URL: https://issues.apache.org/jira/browse/SPARK-3605
Project: Spark
Issue Type: Bug
Components: SQL
[
https://issues.apache.org/jira/browse/SPARK-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14141063#comment-14141063
]
Sandy Ryza commented on SPARK-3573:
---
Currently SchemaRDD does depend on Catalyst. Are y
[
https://issues.apache.org/jira/browse/SPARK-3612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14142006#comment-14142006
]
Sandy Ryza commented on SPARK-3612:
---
Yeah, we should catch this. Will post a patch.
>
[
https://issues.apache.org/jira/browse/SPARK-3577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14142524#comment-14142524
]
Sandy Ryza commented on SPARK-3577:
---
No problem. Yeah, I agree that a spill time metric
Sandy Ryza created SPARK-3642:
-
Summary: Better document the nuances of shared variables
Key: SPARK-3642
URL: https://issues.apache.org/jira/browse/SPARK-3642
Project: Spark
Issue Type: Improveme
[
https://issues.apache.org/jira/browse/SPARK-3622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14143908#comment-14143908
]
Sandy Ryza commented on SPARK-3622:
---
Is this a duplicate of SPARK-2688?
> Provide a cus
[
https://issues.apache.org/jira/browse/SPARK-2142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza resolved SPARK-2142.
---
Resolution: Not a Problem
I ran some tests that indicated that only stop-the-world GC time gets includ
[
https://issues.apache.org/jira/browse/SPARK-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14145829#comment-14145829
]
Sandy Ryza commented on SPARK-3468:
---
This looks like a really cool addition.
> WebUI Ti
Sandy Ryza created SPARK-3682:
-
Summary: Add helpful warnings to the UI
Key: SPARK-3682
URL: https://issues.apache.org/jira/browse/SPARK-3682
Project: Spark
Issue Type: New Feature
Comp
[
https://issues.apache.org/jira/browse/SPARK-2131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza resolved SPARK-2131.
---
Resolution: Duplicate
> Collect per-task filesystem-bytes-read/written metrics
> -
[
https://issues.apache.org/jira/browse/SPARK-3682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza updated SPARK-3682:
--
Target Version/s: 1.3.0 (was: 1.2.0)
> Add helpful warnings to the UI
> --
[
https://issues.apache.org/jira/browse/SPARK-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza resolved SPARK-3422.
---
Resolution: Fixed
> JavaAPISuite.getHadoopInputSplits isn't used anywhere
> --
[
https://issues.apache.org/jira/browse/SPARK-3693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14148303#comment-14148303
]
Sandy Ryza commented on SPARK-3693:
---
Spark's documentation actually makes a note of this
[
https://issues.apache.org/jira/browse/SPARK-3682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza updated SPARK-3682:
--
Description:
Spark has a zillion configuration options and a zillion different things that
can go wrong
[
https://issues.apache.org/jira/browse/SPARK-3682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14148379#comment-14148379
]
Sandy Ryza commented on SPARK-3682:
---
Oops, that should have read "increased".
When a ta
[
https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14158571#comment-14158571
]
Sandy Ryza commented on SPARK-3561:
---
I think there may be somewhat of a misunderstanding
[
https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza updated SPARK-3561:
--
Summary: Decouple Spark's API from its execution engine (was: Native
Hadoop/YARN integration for batch/
[
https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza updated SPARK-3561:
--
Description:
Currently Spark's API is tightly coupled with its backend execution engine.
It could be
[
https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza updated SPARK-3561:
--
Description:
Currently Spark's API is tightly coupled with its backend execution engine.
It could be
[
https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza updated SPARK-3561:
--
Description:
Currently Spark's user-facing API is tightly coupled with its backend execution
engine.
[
https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14158571#comment-14158571
]
Sandy Ryza edited comment on SPARK-3561 at 10/3/14 11:00 PM:
-
[
https://issues.apache.org/jira/browse/SPARK-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159252#comment-14159252
]
Sandy Ryza commented on SPARK-3464:
---
Did you mean to resolve this as "Fixed"?
> Gracefu
[
https://issues.apache.org/jira/browse/SPARK-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159252#comment-14159252
]
Sandy Ryza edited comment on SPARK-3464 at 10/4/14 7:27 PM:
[~
[
https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14160937#comment-14160937
]
Sandy Ryza commented on SPARK-3174:
---
Thanks for posting the detailed design, Andrew. A
[
https://issues.apache.org/jira/browse/SPARK-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza updated SPARK-3797:
--
Summary: Enable running shuffle service in separate process from executor
(was: Integrate shuffle servi
[
https://issues.apache.org/jira/browse/SPARK-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza updated SPARK-3797:
--
Description:
This could either mean
* Running the shuffle service inside the YARN NodeManager as an Auxi
[
https://issues.apache.org/jira/browse/SPARK-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza updated SPARK-3797:
--
Description:
This could either mean
* Running the shuffle service inside the YARN NodeManager as an auxi
[
https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14160960#comment-14160960
]
Sandy Ryza commented on SPARK-3174:
---
bq. for instance, lets say I do some ETL stuff wher
[
https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161048#comment-14161048
]
Sandy Ryza commented on SPARK-3174:
---
Ah, misread.
My opinion is that, for a first cut w
[
https://issues.apache.org/jira/browse/SPARK-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza updated SPARK-3797:
--
Summary: Run the shuffle service inside the YARN NodeManager as an
AuxiliaryService (was: Enable runnin
[
https://issues.apache.org/jira/browse/SPARK-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza updated SPARK-3797:
--
Description: It's also worth considering running the shuffle service in a
YARN container beside the exec
[
https://issues.apache.org/jira/browse/SPARK-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161636#comment-14161636
]
Sandy Ryza commented on SPARK-3797:
---
Not necessarily opposed to this, but wanted to brin
Sandy Ryza created SPARK-3837:
-
Summary: Warn when YARN is killing containers for exceeding memory
limits
Key: SPARK-3837
URL: https://issues.apache.org/jira/browse/SPARK-3837
Project: Spark
Iss
[
https://issues.apache.org/jira/browse/SPARK-3682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza updated SPARK-3682:
--
Attachment: SPARK-3682Design.pdf
Posting an initial design
> Add helpful warnings to the UI
> -
[
https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162685#comment-14162685
]
Sandy Ryza commented on SPARK-3174:
---
bq. Maybe it makes sense to just call it `spark.dy
Sandy Ryza created SPARK-3884:
-
Summary: Don't set SPARK_SUBMIT_DRIVER_MEMORY if deploy mode is
cluster
Key: SPARK-3884
URL: https://issues.apache.org/jira/browse/SPARK-3884
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza updated SPARK-3884:
--
Summary: If deploy mode is cluster, --driver-memory shouldn't apply to
client JVM (was: Don't set SPARK
[
https://issues.apache.org/jira/browse/SPARK-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14165776#comment-14165776
]
Sandy Ryza commented on SPARK-3884:
---
Accidentally assigned this to myself, but others sh
[
https://issues.apache.org/jira/browse/SPARK-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169846#comment-14169846
]
Sandy Ryza commented on SPARK-1209:
---
Definitely worth changing, in my opinion. This has
[
https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169931#comment-14169931
]
Sandy Ryza commented on SPARK-3174:
---
bq. Slow-start is actually not slow at all if we lo
[
https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14171024#comment-14171024
]
Sandy Ryza commented on SPARK-3174:
---
bq. If I understand correctly, your concern with re
[
https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14171024#comment-14171024
]
Sandy Ryza edited comment on SPARK-3174 at 10/14/14 3:03 PM:
-
[
https://issues.apache.org/jira/browse/SPARK-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14171238#comment-14171238
]
Sandy Ryza commented on SPARK-3360:
---
bq. You don't need Vector.multiply(RowMatrix) reall
[
https://issues.apache.org/jira/browse/SPARK-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179631#comment-14179631
]
Sandy Ryza commented on SPARK-2926:
---
[~rxin] did you ever get a chance to try this out?
[
https://issues.apache.org/jira/browse/SPARK-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183173#comment-14183173
]
Sandy Ryza commented on SPARK-3573:
---
Is this still targeted for 1.2?
> Dataset
> --
[
https://issues.apache.org/jira/browse/SPARK-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183174#comment-14183174
]
Sandy Ryza commented on SPARK-1856:
---
Is this work still targeted for 1.2?
> Standardize
[
https://issues.apache.org/jira/browse/SPARK-3461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14186562#comment-14186562
]
Sandy Ryza commented on SPARK-3461:
---
SPARK-2926 could help with this as well.
> Support
Sandy Ryza created SPARK-4136:
-
Summary: Under dynamic allocation, cancel outstanding executor
requests when pending task queue is empty
Key: SPARK-4136
URL: https://issues.apache.org/jira/browse/SPARK-4136
Sandy Ryza created SPARK-4175:
-
Summary: Exception on stage page
Key: SPARK-4175
URL: https://issues.apache.org/jira/browse/SPARK-4175
Project: Spark
Issue Type: Bug
Affects Versions: 1.2.0
[
https://issues.apache.org/jira/browse/SPARK-4016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14192604#comment-14192604
]
Sandy Ryza commented on SPARK-4016:
---
It looks like after this change, stage-level summar
[
https://issues.apache.org/jira/browse/SPARK-4016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14192609#comment-14192609
]
Sandy Ryza commented on SPARK-4016:
---
Also, it looks like this can cause an exception: SP
Sandy Ryza created SPARK-4178:
-
Summary: Hadoop input metrics ignore bytes read in RecordReader
instantiation
Key: SPARK-4178
URL: https://issues.apache.org/jira/browse/SPARK-4178
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14192773#comment-14192773
]
Sandy Ryza commented on SPARK-4178:
---
Thanks [~kostas] for noticing this.
> Hadoop input
[
https://issues.apache.org/jira/browse/SPARK-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14601593#comment-14601593
]
Sandy Ryza commented on SPARK-8623:
---
Looking into it
> Some queries in spark-sql lead t
[
https://issues.apache.org/jira/browse/SPARK-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14601644#comment-14601644
]
Sandy Ryza commented on SPARK-8623:
---
I took a look at the line numbers and it seems like
[
https://issues.apache.org/jira/browse/SPARK-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14603243#comment-14603243
]
Sandy Ryza commented on SPARK-8623:
---
Am able to reproduce this locally. Looking into th
[
https://issues.apache.org/jira/browse/SPARK-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14603694#comment-14603694
]
Sandy Ryza commented on SPARK-8623:
---
Figured out the issue - my patch omitted registerin
[
https://issues.apache.org/jira/browse/SPARK-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza updated SPARK-8623:
--
Component/s: (was: SQL)
Spark Core
> Some queries in spark-sql lead to NullPointerE
[
https://issues.apache.org/jira/browse/SPARK-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza updated SPARK-8623:
--
Summary: Hadoop RDDs fail to properly serialize configuration (was: Some
queries in spark-sql lead to N
[
https://issues.apache.org/jira/browse/SPARK-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza reassigned SPARK-8623:
-
Assignee: Sandy Ryza
> Some queries in spark-sql lead to NullPointerException when using Yarn
> -
[
https://issues.apache.org/jira/browse/SPARK-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14980717#comment-14980717
]
Sandy Ryza commented on SPARK-2089:
---
My opinion is that we should be moving towards dyna
[
https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15022634#comment-15022634
]
Sandy Ryza commented on SPARK-:
---
[~nchammas] it's not clear that it makes sense to a
Sandy Ryza created SPARK-2084:
-
Summary: Mention SPARK_JAR in env var section on configuration page
Key: SPARK-2084
URL: https://issues.apache.org/jira/browse/SPARK-2084
Project: Spark
Issue Type
Sandy Ryza created SPARK-2089:
-
Summary: With YARN, preferredNodeLocalityData isn't honored
Key: SPARK-2089
URL: https://issues.apache.org/jira/browse/SPARK-2089
Project: Spark
Issue Type: Bug
Sandy Ryza created SPARK-2099:
-
Summary: Report metrics for running tasks
Key: SPARK-2099
URL: https://issues.apache.org/jira/browse/SPARK-2099
Project: Spark
Issue Type: Improvement
Affects
1 - 100 of 536 matches
Mail list logo
