[jira] [Comment Edited] (SPARK-21549) Spark fails to complete job correctly in case of OutputFormat which do not write into hdfs

[ https://issues.apache.org/jira/browse/SPARK-21549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105630#comment-16105630 ] Mridul Muralidharan edited comment on SPARK-21549 at 7/28/17 10:14 PM:

[jira] [Commented] (SPARK-21549) Spark fails to complete job correctly in case of OutputFormat which do not write into hdfs

[ https://issues.apache.org/jira/browse/SPARK-21549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105630#comment-16105630 ] Mridul Muralidharan commented on SPARK-21549: - This affects both mapred

[jira] [Commented] (SPARK-20589) Allow limiting task concurrency per stage

[ https://issues.apache.org/jira/browse/SPARK-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995550#comment-15995550 ] Mridul Muralidharan commented on SPARK-20589: - coalasce with shuffle=false might be a

[jira] [Commented] (SPARK-20480) FileFormatWriter hides FetchFailedException from scheduler

[ https://issues.apache.org/jira/browse/SPARK-20480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985575#comment-15985575 ] Mridul Muralidharan commented on SPARK-20480: - Shouldn't fix for SPARK-19276 by [~imranr] not

[jira] [Comment Edited] (SPARK-20205) DAGScheduler posts SparkListenerStageSubmitted before updating stage

[ https://issues.apache.org/jira/browse/SPARK-20205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15954387#comment-15954387 ] Mridul Muralidharan edited comment on SPARK-20205 at 4/4/17 12:15 AM:

[jira] [Commented] (SPARK-20205) DAGScheduler posts SparkListenerStageSubmitted before updating stage

[ https://issues.apache.org/jira/browse/SPARK-20205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15954387#comment-15954387 ] Mridul Muralidharan commented on SPARK-20205: - For history server that will fail - good

[jira] [Commented] (SPARK-20205) DAGScheduler posts SparkListenerStageSubmitted before updating stage

[ https://issues.apache.org/jira/browse/SPARK-20205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15954348#comment-15954348 ] Mridul Muralidharan commented on SPARK-20205: - bq. I wouldn't say incorrect; at worst it's

[jira] [Commented] (SPARK-20205) DAGScheduler posts SparkListenerStageSubmitted before updating stage

[ https://issues.apache.org/jira/browse/SPARK-20205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15954333#comment-15954333 ] Mridul Muralidharan commented on SPARK-20205: - This is nasty ! This means submissionTime will

[jira] [Commented] (SPARK-19796) taskScheduler fails serializing long statements received by thrift server

[ https://issues.apache.org/jira/browse/SPARK-19796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893584#comment-15893584 ] Mridul Muralidharan commented on SPARK-19796: - I would not prefer (b) - if we are worried

[jira] [Commented] (SPARK-5159) Thrift server does not respect hive.server2.enable.doAs=true

[ https://issues.apache.org/jira/browse/SPARK-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884979#comment-15884979 ] Mridul Muralidharan commented on SPARK-5159: For 1.6.x and earlier, we found that doAs

[jira] [Commented] (SPARK-19698) Race condition in stale attempt task completion vs current attempt task completion when task is doing persistent state changes

[ https://issues.apache.org/jira/browse/SPARK-19698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881991#comment-15881991 ] Mridul Muralidharan commented on SPARK-19698: - Depending on ordering and semantics of task

[jira] [Comment Edited] (SPARK-19698) Race condition in stale attempt task completion vs current attempt task completion when task is doing persistent state changes

[ https://issues.apache.org/jira/browse/SPARK-19698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881991#comment-15881991 ] Mridul Muralidharan edited comment on SPARK-19698 at 2/24/17 5:25 AM:

[jira] [Assigned] (SPARK-19244) Sort MemoryConsumers according to their memory usage when spilling

[ https://issues.apache.org/jira/browse/SPARK-19244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-19244: --- Assignee: Liang-Chi Hsieh > Sort MemoryConsumers according to their memory

[jira] [Resolved] (SPARK-19244) Sort MemoryConsumers according to their memory usage when spilling

[ https://issues.apache.org/jira/browse/SPARK-19244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-19244. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request

[jira] [Commented] (SPARK-19143) API in Spark for distributing new delegation tokens (Improve delegation token handling in secure clusters)

[ https://issues.apache.org/jira/browse/SPARK-19143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816026#comment-15816026 ] Mridul Muralidharan commented on SPARK-19143: - bq. You say "we added", you are saying you

[jira] [Commented] (SPARK-19143) API in Spark for distributing new delegation tokens (Improve delegation token handling in secure clusters)

[ https://issues.apache.org/jira/browse/SPARK-19143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814408#comment-15814408 ] Mridul Muralidharan commented on SPARK-19143: - As part of an unrelated work (though

[jira] [Commented] (SPARK-18886) Delay scheduling should not delay some executors indefinitely if one task is scheduled before delay timeout

[ https://issues.apache.org/jira/browse/SPARK-18886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764876#comment-15764876 ] Mridul Muralidharan commented on SPARK-18886: - I am not sure what is described will work as

[jira] [Commented] (SPARK-18886) Delay scheduling should not delay some executors indefinitely if one task is scheduled before delay timeout

[ https://issues.apache.org/jira/browse/SPARK-18886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756179#comment-15756179 ] Mridul Muralidharan commented on SPARK-18886: - - Delay 'using up' all resources - another

[jira] [Comment Edited] (SPARK-18886) Delay scheduling should not delay some executors indefinitely if one task is scheduled before delay timeout

[ https://issues.apache.org/jira/browse/SPARK-18886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15752579#comment-15752579 ] Mridul Muralidharan edited comment on SPARK-18886 at 12/15/16 9:35 PM:

[jira] [Commented] (SPARK-18886) Delay scheduling should not delay some executors indefinitely if one task is scheduled before delay timeout

[ https://issues.apache.org/jira/browse/SPARK-18886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15752579#comment-15752579 ] Mridul Muralidharan commented on SPARK-18886: - [~imranr] For almost all cases, delay

[jira] [Commented] (SPARK-18886) Delay scheduling should not delay some executors indefinitely if one task is scheduled before delay timeout

[ https://issues.apache.org/jira/browse/SPARK-18886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15752426#comment-15752426 ] Mridul Muralidharan commented on SPARK-18886: - Spark scheduler can be suboptimal for a lot

[jira] [Commented] (SPARK-16554) Spark should kill executors when they are blacklisted

[ https://issues.apache.org/jira/browse/SPARK-16554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703851#comment-15703851 ] Mridul Muralidharan commented on SPARK-16554: - It would also be good if we can also move

[jira] [Resolved] (SPARK-16759) Spark expose an API to pass in Caller Context into it

[ https://issues.apache.org/jira/browse/SPARK-16759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-16759. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request

[jira] [Updated] (SPARK-16759) Spark expose an API to pass in Caller Context into it

[ https://issues.apache.org/jira/browse/SPARK-16759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-16759: Assignee: Weiqing Yang > Spark expose an API to pass in Caller Context into it >

[jira] [Closed] (SPARK-18250) Minor issues with utf8string observed

[ https://issues.apache.org/jira/browse/SPARK-18250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan closed SPARK-18250. --- Resolution: Later > Minor issues with utf8string observed >

[jira] [Updated] (SPARK-18250) Minor issues with utf8string observed

[ https://issues.apache.org/jira/browse/SPARK-18250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-18250: Description: A few minor issues observed with utf8 which did not match the

[jira] [Created] (SPARK-18250) Minor issues with utf8string observed

Mridul Muralidharan created SPARK-18250: --- Summary: Minor issues with utf8string observed Key: SPARK-18250 URL: https://issues.apache.org/jira/browse/SPARK-18250 Project: Spark Issue

[jira] [Closed] (SPARK-18008) Support skipping test compilation

[ https://issues.apache.org/jira/browse/SPARK-18008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan closed SPARK-18008. --- Resolution: Won't Fix Closing jira due to pushback in commiting it. See pr for

[jira] [Updated] (SPARK-18008) Support skipping test compilation

[ https://issues.apache.org/jira/browse/SPARK-18008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-18008: Description: Add support for skipping compilation of test code through

[jira] [Updated] (SPARK-18008) Support skipping test compilation

[ https://issues.apache.org/jira/browse/SPARK-18008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-18008: Summary: Support skipping test compilation (was: Support test compilation and

[jira] [Created] (SPARK-18008) Support test compilation and skipping javadoc generation

Mridul Muralidharan created SPARK-18008: --- Summary: Support test compilation and skipping javadoc generation Key: SPARK-18008 URL: https://issues.apache.org/jira/browse/SPARK-18008 Project: Spark

[jira] [Updated] (SPARK-17637) Packed scheduling for Spark tasks across executors

[ https://issues.apache.org/jira/browse/SPARK-17637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-17637: Affects Version/s: 2.1.0 > Packed scheduling for Spark tasks across executors >

[jira] [Resolved] (SPARK-17637) Packed scheduling for Spark tasks across executors

[ https://issues.apache.org/jira/browse/SPARK-17637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-17637. - Resolution: Fixed Assignee: Zhan Zhang Target Version/s:

[jira] [Updated] (SPARK-17637) Packed scheduling for Spark tasks across executors

[ https://issues.apache.org/jira/browse/SPARK-17637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-17637: Fix Version/s: 2.1.0 > Packed scheduling for Spark tasks across executors >

[jira] [Commented] (SPARK-17064) Reconsider spark.job.interruptOnCancel

[ https://issues.apache.org/jira/browse/SPARK-17064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547073#comment-15547073 ] Mridul Muralidharan commented on SPARK-17064: - I agree, interrupt'ing a thread can (and

[jira] [Closed] (SPARK-7313) Allow for configuring max_samples in range partitioner.

[ https://issues.apache.org/jira/browse/SPARK-7313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan closed SPARK-7313. -- Resolution: Won't Fix Not sure if still relevant, I opened it for an earlier project

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15204939#comment-15204939 ] Mridul Muralidharan commented on SPARK-1239: [~tgraves] For the last part (waiting bit) - why

[jira] [Commented] (SPARK-11293) Spillable collections leak shuffle memory

[ https://issues.apache.org/jira/browse/SPARK-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15128935#comment-15128935 ] Mridul Muralidharan commented on SPARK-11293: - Not iterating to the end has a bunch of issues

[jira] [Commented] (SPARK-6166) Add config to limit number of concurrent outbound connections for shuffle fetch

[ https://issues.apache.org/jira/browse/SPARK-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15103065#comment-15103065 ] Mridul Muralidharan commented on SPARK-6166: We actually dont care about number of sockets or

[jira] [Commented] (SPARK-6166) Add config to limit number of concurrent outbound connections for shuffle fetch

[ https://issues.apache.org/jira/browse/SPARK-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15082354#comment-15082354 ] Mridul Muralidharan commented on SPARK-6166: /CC [~tgraves] who might be interested in this.

[jira] [Commented] (SPARK-6166) Add config to limit number of concurrent outbound connections for shuffle fetch

[ https://issues.apache.org/jira/browse/SPARK-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15082191#comment-15082191 ] Mridul Muralidharan commented on SPARK-6166: The resource usage incurred per node as part of

[jira] [Commented] (SPARK-11801) Notify driver when OOM is thrown before executor JVM is killed

[ https://issues.apache.org/jira/browse/SPARK-11801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15037869#comment-15037869 ] Mridul Muralidharan commented on SPARK-11801: - I have preference for (a) because I am not

[jira] [Commented] (SPARK-11801) Notify driver when OOM is thrown before executor JVM is killed

[ https://issues.apache.org/jira/browse/SPARK-11801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15034914#comment-15034914 ] Mridul Muralidharan commented on SPARK-11801: - There are few aspects here : a) A race

[jira] [Comment Edited] (SPARK-11801) Notify driver when OOM is thrown before executor JVM is killed

[ https://issues.apache.org/jira/browse/SPARK-11801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15034914#comment-15034914 ] Mridul Muralidharan edited comment on SPARK-11801 at 12/2/15 12:01 AM:

[jira] [Commented] (SPARK-11801) Notify driver when OOM is thrown before executor JVM is killed

[ https://issues.apache.org/jira/browse/SPARK-11801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15032736#comment-15032736 ] Mridul Muralidharan commented on SPARK-11801: - To give background: It was a conscious

[jira] [Commented] (SPARK-2089) With YARN, preferredNodeLocalityData isn't honored

[ https://issues.apache.org/jira/browse/SPARK-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14620103#comment-14620103 ] Mridul Muralidharan commented on SPARK-2089: Let us keep both functionality

[jira] [Comment Edited] (SPARK-2089) With YARN, preferredNodeLocalityData isn't honored

[ https://issues.apache.org/jira/browse/SPARK-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14620103#comment-14620103 ] Mridul Muralidharan edited comment on SPARK-2089 at 7/9/15 8:36 AM:

[jira] [Assigned] (SPARK-8297) Scheduler backend is not notified in case node fails in YARN

[ https://issues.apache.org/jira/browse/SPARK-8297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-8297: -- Assignee: Mridul Muralidharan Scheduler backend is not notified in case node

[jira] [Commented] (SPARK-8297) Scheduler backend is not notified in case node fails in YARN

[ https://issues.apache.org/jira/browse/SPARK-8297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14583560#comment-14583560 ] Mridul Muralidharan commented on SPARK-8297: I am not sure why the spark

[jira] [Commented] (SPARK-8297) Scheduler backend is not notified in case node fails in YARN

[ https://issues.apache.org/jira/browse/SPARK-8297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14581713#comment-14581713 ] Mridul Muralidharan commented on SPARK-8297: kill -9 is not sufficient -

[jira] [Created] (SPARK-8297) Scheduler backend is not notified in case node fails in YARN

Mridul Muralidharan created SPARK-8297: -- Summary: Scheduler backend is not notified in case node fails in YARN Key: SPARK-8297 URL: https://issues.apache.org/jira/browse/SPARK-8297 Project:

[jira] [Updated] (SPARK-8297) Scheduler backend is not notified in case node fails in YARN

[ https://issues.apache.org/jira/browse/SPARK-8297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-8297: --- Assignee: (was: Mridul Muralidharan) Scheduler backend is not notified in case

[jira] [Commented] (SPARK-8297) Scheduler backend is not notified in case node fails in YARN

[ https://issues.apache.org/jira/browse/SPARK-8297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14581038#comment-14581038 ] Mridul Muralidharan commented on SPARK-8297: Spark on mesos handles this

[jira] [Updated] (SPARK-8297) Scheduler backend is not notified in case node fails in YARN

[ https://issues.apache.org/jira/browse/SPARK-8297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-8297: --- Affects Version/s: 1.5.0 Scheduler backend is not notified in case node fails in

[jira] [Assigned] (SPARK-7323) Use insertAll instead of individual insert while merging combiners

[ https://issues.apache.org/jira/browse/SPARK-7323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-7323: -- Assignee: Mridul Muralidharan Use insertAll instead of individual insert

[jira] [Created] (SPARK-7323) Use insertAll instead of individual insert while merging combiners

Mridul Muralidharan created SPARK-7323: -- Summary: Use insertAll instead of individual insert while merging combiners Key: SPARK-7323 URL: https://issues.apache.org/jira/browse/SPARK-7323

[jira] [Created] (SPARK-7317) ShuffleHandle needs to be exposed

Mridul Muralidharan created SPARK-7317: -- Summary: ShuffleHandle needs to be exposed Key: SPARK-7317 URL: https://issues.apache.org/jira/browse/SPARK-7317 Project: Spark Issue Type:

[jira] [Created] (SPARK-7313) Allow for configuring max_samples in range partitioner.

Mridul Muralidharan created SPARK-7313: -- Summary: Allow for configuring max_samples in range partitioner. Key: SPARK-7313 URL: https://issues.apache.org/jira/browse/SPARK-7313 Project: Spark

[jira] [Closed] (SPARK-6168) Expose some of the collection classes as DeveloperApi

[ https://issues.apache.org/jira/browse/SPARK-6168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan closed SPARK-6168. -- Resolution: Won't Fix Expose some of the collection classes as DeveloperApi

[jira] [Updated] (SPARK-1476) 2GB limit in spark for blocks

[ https://issues.apache.org/jira/browse/SPARK-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-1476: --- Assignee: (was: Mridul Muralidharan) 2GB limit in spark for blocks

[jira] [Updated] (SPARK-2532) Fix issues with consolidated shuffle

[ https://issues.apache.org/jira/browse/SPARK-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-2532: --- Assignee: (was: Mridul Muralidharan) Fix issues with consolidated shuffle

[jira] [Reopened] (SPARK-6168) Expose some of the collection classes as DeveloperApi

[ https://issues.apache.org/jira/browse/SPARK-6168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reopened SPARK-6168: Assignee: Mridul Muralidharan Expose some of the collection classes as

[jira] [Commented] (SPARK-6168) Expose some of the collection classes as DeveloperApi

[ https://issues.apache.org/jira/browse/SPARK-6168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380759#comment-14380759 ] Mridul Muralidharan commented on SPARK-6168: Looks like I did not get

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352377#comment-14352377 ] Mridul Muralidharan commented on SPARK-1239: Hitting akka framesize for map

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352284#comment-14352284 ] Mridul Muralidharan commented on SPARK-1239: [~pwendell] Is there any update

[jira] [Commented] (SPARK-6169) Shuffle based join

[ https://issues.apache.org/jira/browse/SPARK-6169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348995#comment-14348995 ] Mridul Muralidharan commented on SPARK-6169: I have not done too much research

[jira] [Commented] (SPARK-4879) Missing output partitions after job completes with speculative execution

[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348987#comment-14348987 ] Mridul Muralidharan commented on SPARK-4879: [~joshrosen] The former - the

[jira] [Updated] (SPARK-4879) Missing output partitions after job completes with speculative execution

[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-4879: --- Affects Version/s: 1.3.0 Missing output partitions after job completes with

[jira] [Commented] (SPARK-4879) Missing output partitions after job completes with speculative execution

[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348522#comment-14348522 ] Mridul Muralidharan commented on SPARK-4879: With 1.3 RC, we are still seeing

[jira] [Commented] (SPARK-4879) Missing output partitions after job completes with speculative execution

[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348527#comment-14348527 ] Mridul Muralidharan commented on SPARK-4879: Just to add - we are seeing this

[jira] [Updated] (SPARK-6166) Add config to limit number of concurrent outbound connections for shuffle fetch

[ https://issues.apache.org/jira/browse/SPARK-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-6166: --- Priority: Minor (was: Major) Add config to limit number of concurrent outbound

[jira] [Created] (SPARK-6166) Add config to limit number of concurrent outbound connections for shuffle fetch

Mridul Muralidharan created SPARK-6166: -- Summary: Add config to limit number of concurrent outbound connections for shuffle fetch Key: SPARK-6166 URL: https://issues.apache.org/jira/browse/SPARK-6166

[jira] [Created] (SPARK-6168) Expose some of the collection classes as DeveloperApi

Mridul Muralidharan created SPARK-6168: -- Summary: Expose some of the collection classes as DeveloperApi Key: SPARK-6168 URL: https://issues.apache.org/jira/browse/SPARK-6168 Project: Spark

[jira] [Updated] (SPARK-6168) Expose some of the collection classes as DeveloperApi

[ https://issues.apache.org/jira/browse/SPARK-6168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-6168: --- Description: It would be very useful to mark some of the collection and util classes

[jira] [Created] (SPARK-6169) Shuffle based join

Mridul Muralidharan created SPARK-6169: -- Summary: Shuffle based join Key: SPARK-6169 URL: https://issues.apache.org/jira/browse/SPARK-6169 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-6170) Add support for skew join

Mridul Muralidharan created SPARK-6170: -- Summary: Add support for skew join Key: SPARK-6170 URL: https://issues.apache.org/jira/browse/SPARK-6170 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-6165) Aggregate and reduce should spool to disk and complete

Mridul Muralidharan created SPARK-6165: -- Summary: Aggregate and reduce should spool to disk and complete Key: SPARK-6165 URL: https://issues.apache.org/jira/browse/SPARK-6165 Project: Spark

[jira] [Updated] (SPARK-6165) Aggregate and reduce should be able to work with very large number of tasks.

[ https://issues.apache.org/jira/browse/SPARK-6165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-6165: --- Summary: Aggregate and reduce should be able to work with very large number of tasks.

[jira] [Commented] (SPARK-3889) JVM dies with SIGBUS, resulting in ConnectionManager failed ACK

[ https://issues.apache.org/jira/browse/SPARK-3889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341997#comment-14341997 ] Mridul Muralidharan commented on SPARK-3889: [~adav] I have seen this a lot in

[jira] [Commented] (SPARK-6050) Spark on YARN does not work --executor-cores is specified

[ https://issues.apache.org/jira/browse/SPARK-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340379#comment-14340379 ] Mridul Muralidharan commented on SPARK-6050: [~tgraves] You are right, cpu

[jira] [Comment Edited] (SPARK-6050) Spark on YARN does not work --executor-cores is specified

[ https://issues.apache.org/jira/browse/SPARK-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14339686#comment-14339686 ] Mridul Muralidharan edited comment on SPARK-6050 at 2/27/15 5:24 AM:

[jira] [Commented] (SPARK-6050) Spark on YARN does not work --executor-cores is specified

[ https://issues.apache.org/jira/browse/SPARK-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14339798#comment-14339798 ] Mridul Muralidharan commented on SPARK-6050: With more verbose debug added,

[jira] [Created] (SPARK-6050) Spark on YARN does not work --executor-cores is specified

Mridul Muralidharan created SPARK-6050: -- Summary: Spark on YARN does not work --executor-cores is specified Key: SPARK-6050 URL: https://issues.apache.org/jira/browse/SPARK-6050 Project: Spark

[jira] [Comment Edited] (SPARK-6050) Spark on YARN does not work --executor-cores is specified

[ https://issues.apache.org/jira/browse/SPARK-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14339686#comment-14339686 ] Mridul Muralidharan edited comment on SPARK-6050 at 2/27/15 3:50 AM:

[jira] [Commented] (SPARK-6050) Spark on YARN does not work --executor-cores is specified

[ https://issues.apache.org/jira/browse/SPARK-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14339686#comment-14339686 ] Mridul Muralidharan commented on SPARK-6050: Thanks to [~tgraves] for helping

[jira] [Commented] (SPARK-4030) `destroy` method in Broadcast should be public

[ https://issues.apache.org/jira/browse/SPARK-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14178162#comment-14178162 ] Mridul Muralidharan commented on SPARK-4030: We have also needed to use

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173413#comment-14173413 ] Mridul Muralidharan commented on SPARK-3948: Not exactly, what I was

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173489#comment-14173489 ] Mridul Muralidharan commented on SPARK-3948: Damn, this sucks : the

[jira] [Comment Edited] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173489#comment-14173489 ] Mridul Muralidharan edited comment on SPARK-3948 at 10/16/14 7:39 AM:

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172116#comment-14172116 ] Mridul Muralidharan commented on SPARK-3948: [~joshrosen] Assuming there are

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172134#comment-14172134 ] Mridul Muralidharan commented on SPARK-3948: [~jerryshao] Just to clarify,

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172225#comment-14172225 ] Mridul Muralidharan commented on SPARK-3948: That is weird, I tried a stripped

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172226#comment-14172226 ] Mridul Muralidharan commented on SPARK-3948: Note, t is just some file with a

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172585#comment-14172585 ] Mridul Muralidharan commented on SPARK-3948: [~jerryshao] great work ! I

[jira] [Commented] (SPARK-3889) JVM dies with SIGBUS, resulting in ConnectionManager failed ACK

[ https://issues.apache.org/jira/browse/SPARK-3889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167303#comment-14167303 ] Mridul Muralidharan commented on SPARK-3889: The status says fixed - what was

[jira] [Commented] (SPARK-3785) Support off-loading computations to a GPU

[ https://issues.apache.org/jira/browse/SPARK-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14163326#comment-14163326 ] Mridul Muralidharan commented on SPARK-3785: [~sowen] We had prototyped a

[jira] [Commented] (SPARK-3561) Allow for pluggable execution contexts in Spark

[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14163351#comment-14163351 ] Mridul Muralidharan commented on SPARK-3561: [~pwendell] If I understood the

[jira] [Commented] (SPARK-3561) Allow for pluggable execution contexts in Spark

[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14163376#comment-14163376 ] Mridul Muralidharan commented on SPARK-3561: [~ozhurakousky] I think the

[jira] [Commented] (SPARK-3847) Enum.hashCode is only consistent within the same JVM

[ https://issues.apache.org/jira/browse/SPARK-3847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14163407#comment-14163407 ] Mridul Muralidharan commented on SPARK-3847: Wow, nice bug ! This is

[jira] [Commented] (SPARK-3561) Allow for pluggable execution contexts in Spark

[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14164095#comment-14164095 ] Mridul Muralidharan commented on SPARK-3561: I agree with [~pwendell] that it

<    1   2   3   4   5   6   >