[jira] [Commented] (SPARK-12345) Mesos cluster mode is broken

[ https://issues.apache.org/jira/browse/SPARK-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065047#comment-15065047 ] Andrew Or commented on SPARK-12345: --- For those who are following: There are 4 patches related to this

[jira] [Updated] (SPARK-12365) Use ShutdownHookManager where Runtime.getRuntime.addShutdownHook() is called

[ https://issues.apache.org/jira/browse/SPARK-12365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12365: -- Fix Version/s: (was: 1.6.1) > Use ShutdownHookManager where Runtime.getRuntime.addShutdownHook()

[jira] [Updated] (SPARK-12365) Use ShutdownHookManager where Runtime.getRuntime.addShutdownHook() is called

[ https://issues.apache.org/jira/browse/SPARK-12365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12365: -- Target Version/s: 2.0.0 (was: 1.6.1, 2.0.0) > Use ShutdownHookManager where

[jira] [Updated] (SPARK-12390) Clean up unused serializer parameter in BlockManager

[ https://issues.apache.org/jira/browse/SPARK-12390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12390: -- Fix Version/s: 1.6.1 > Clean up unused serializer parameter in BlockManager >

[jira] [Resolved] (SPARK-12390) Clean up unused serializer parameter in BlockManager

[ https://issues.apache.org/jira/browse/SPARK-12390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12390. --- Resolution: Fixed > Clean up unused serializer parameter in BlockManager >

[jira] [Assigned] (SPARK-12390) Clean up unused serializer parameter in BlockManager

[ https://issues.apache.org/jira/browse/SPARK-12390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reassigned SPARK-12390: - Assignee: Andrew Or (was: Apache Spark) > Clean up unused serializer parameter in BlockManager

[jira] [Created] (SPARK-12415) Do not use closure serializer to serialize task result

Andrew Or created SPARK-12415: - Summary: Do not use closure serializer to serialize task result Key: SPARK-12415 URL: https://issues.apache.org/jira/browse/SPARK-12415 Project: Spark Issue Type:

[jira] [Created] (SPARK-12414) Remove closure serializer

Andrew Or created SPARK-12414: - Summary: Remove closure serializer Key: SPARK-12414 URL: https://issues.apache.org/jira/browse/SPARK-12414 Project: Spark Issue Type: Bug Components:

[jira] [Updated] (SPARK-12345) Mesos cluster mode is broken when SPARK_HOME is set

[ https://issues.apache.org/jira/browse/SPARK-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12345: -- Summary: Mesos cluster mode is broken when SPARK_HOME is set (was: Mesos cluster mode is broken) >

[jira] [Updated] (SPARK-12345) Mesos cluster mode is broken

[ https://issues.apache.org/jira/browse/SPARK-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12345: -- Summary: Mesos cluster mode is broken (was: Mesos cluster mode is broken when SPARK_HOME is set) >

[jira] [Resolved] (SPARK-12345) Mesos cluster mode is broken

[ https://issues.apache.org/jira/browse/SPARK-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12345. --- Resolution: Fixed Assignee: Luc Bourlier (was: Apache Spark) > Mesos cluster mode is broken >

[jira] [Updated] (SPARK-12345) Mesos cluster mode is broken

[ https://issues.apache.org/jira/browse/SPARK-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12345: -- Fix Version/s: 1.6.0 > Mesos cluster mode is broken > > >

[jira] [Updated] (SPARK-12345) Mesos cluster mode is broken

[ https://issues.apache.org/jira/browse/SPARK-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12345: -- Target Version/s: 1.6.0 (was: 1.6.1) > Mesos cluster mode is broken > >

[jira] [Resolved] (SPARK-12365) Use ShutdownHookManager where Runtime.getRuntime.addShutdownHook() is called

[ https://issues.apache.org/jira/browse/SPARK-12365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12365. --- Resolution: Fixed Assignee: Ted Yu (was: Apache Spark) Fix Version/s: 2.0.0

[jira] [Resolved] (SPARK-12186) stage web URI will redirect to the wrong location if it is the first URI from the application to be requested from the history server

[ https://issues.apache.org/jira/browse/SPARK-12186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12186. --- Resolution: Fixed Fix Version/s: 1.6.0 Target Version/s: 1.6.0 > stage web URI will

[jira] [Updated] (SPARK-12186) stage web URI will redirect to the wrong location if it is the first URI from the application to be requested from the history server

[ https://issues.apache.org/jira/browse/SPARK-12186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12186: -- Fix Version/s: (was: 1.6.0) 2.0.0 1.6.1 > stage web URI will

[jira] [Updated] (SPARK-12186) stage web URI will redirect to the wrong location if it is the first URI from the application to be requested from the history server

[ https://issues.apache.org/jira/browse/SPARK-12186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12186: -- Target Version/s: 1.6.1, 2.0.0 (was: 1.6.0) > stage web URI will redirect to the wrong location if it

[jira] [Updated] (SPARK-12186) stage web URI will redirect to the wrong location if it is the first URI from the application to be requested from the history server

[ https://issues.apache.org/jira/browse/SPARK-12186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12186: -- Assignee: Rohit Agarwal > stage web URI will redirect to the wrong location if it is the first URI

[jira] [Resolved] (SPARK-10248) DAGSchedulerSuite should check there were no errors in EventProcessLoop

[ https://issues.apache.org/jira/browse/SPARK-10248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-10248. --- Resolution: Fixed Assignee: Imran Rashid Fix Version/s: 2.0.0

[jira] [Updated] (SPARK-12345) Mesos cluster mode is broken

[ https://issues.apache.org/jira/browse/SPARK-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12345: -- Assignee: Timothy Chen (was: Luc Bourlier) > Mesos cluster mode is broken >

[jira] [Resolved] (SPARK-12386) Setting "spark.executor.port" leads to NPE in SparkEnv

[ https://issues.apache.org/jira/browse/SPARK-12386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12386. --- Resolution: Fixed Assignee: Marcelo Vanzin (was: Apache Spark) Fix

[jira] [Updated] (SPARK-12390) Clean up unused serializer parameter in BlockManager

[ https://issues.apache.org/jira/browse/SPARK-12390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12390: -- Fix Version/s: 2.0.0 > Clean up unused serializer parameter in BlockManager >

[jira] [Created] (SPARK-12390) Clean up unused serializer parameter in BlockManager

Andrew Or created SPARK-12390: - Summary: Clean up unused serializer parameter in BlockManager Key: SPARK-12390 URL: https://issues.apache.org/jira/browse/SPARK-12390 Project: Spark Issue Type:

[jira] [Commented] (SPARK-9042) Spark SQL incompatibility if security is enforced on the Hive warehouse

[ https://issues.apache.org/jira/browse/SPARK-9042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060208#comment-15060208 ] Andrew Ray commented on SPARK-9042: --- Sean, I think there are a couple issues going on here. In my

[jira] [Created] (SPARK-12345) Mesos cluster mode is broken

Andrew Or created SPARK-12345: - Summary: Mesos cluster mode is broken Key: SPARK-12345 URL: https://issues.apache.org/jira/browse/SPARK-12345 Project: Spark Issue Type: Bug Components:

[jira] [Updated] (SPARK-10250) Scala PairRDDFunctions.groupByKey() should be fault-tolerant of single large groups

[ https://issues.apache.org/jira/browse/SPARK-10250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10250: -- Summary: Scala PairRDDFunctions.groupByKey() should be fault-tolerant of single large groups (was:

[jira] [Updated] (SPARK-12345) Mesos cluster mode is broken

[ https://issues.apache.org/jira/browse/SPARK-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12345: -- Target Version/s: 1.6.1 (was: 1.6.0) > Mesos cluster mode is broken > >

[jira] [Resolved] (SPARK-12130) Replace shuffleManagerClass with shortShuffleMgrNames in ExternalShuffleBlockResolver

[ https://issues.apache.org/jira/browse/SPARK-12130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12130. --- Resolution: Fixed Assignee: Lianhui Wang Fix Version/s: 2.0.0 Target

[jira] [Updated] (SPARK-12130) Replace shuffleManagerClass with shortShuffleMgrNames in ExternalShuffleBlockResolver

[ https://issues.apache.org/jira/browse/SPARK-12130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12130: -- Issue Type: Improvement (was: Bug) > Replace shuffleManagerClass with shortShuffleMgrNames in >

[jira] [Resolved] (SPARK-9886) Validate usages of Runtime.getRuntime.addShutdownHook

[ https://issues.apache.org/jira/browse/SPARK-9886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-9886. -- Resolution: Fixed Assignee: (was: Apache Spark) Fix Version/s: 2.0.0

[jira] [Updated] (SPARK-10477) using DSL in ColumnPruningSuite to improve readablity

[ https://issues.apache.org/jira/browse/SPARK-10477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10477: -- Component/s: Tests > using DSL in ColumnPruningSuite to improve readablity >

[jira] [Resolved] (SPARK-4117) Spark on Yarn handle AM being told command from RM

[ https://issues.apache.org/jira/browse/SPARK-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-4117. -- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > Spark on Yarn handle

[jira] [Resolved] (SPARK-4514) SparkContext localProperties does not inherit property updates across thread reuse

[ https://issues.apache.org/jira/browse/SPARK-4514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-4514. -- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 1.2.1, 1.1.2 (was: 1.1.2,

[jira] [Updated] (SPARK-9026) SimpleFutureAction.onComplete should not tie up a separate thread for each callback

[ https://issues.apache.org/jira/browse/SPARK-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-9026: - Assignee: Richard W. Eggert II (was: Josh Rosen) > SimpleFutureAction.onComplete should not tie up a

[jira] [Updated] (SPARK-4514) SparkContext localProperties does not inherit property updates across thread reuse

[ https://issues.apache.org/jira/browse/SPARK-4514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4514: - Assignee: Richard W. Eggert II (was: Josh Rosen) > SparkContext localProperties does not inherit

[jira] [Updated] (SPARK-4117) Spark on Yarn handle AM being told command from RM

[ https://issues.apache.org/jira/browse/SPARK-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4117: - Assignee: Devaraj K > Spark on Yarn handle AM being told command from RM >

[jira] [Resolved] (SPARK-12105) Add a DataFrame.show() with argument for output PrintStream

[ https://issues.apache.org/jira/browse/SPARK-12105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12105. --- Resolution: Fixed Assignee: Jean-Baptiste Onofré Fix Version/s: 2.0.0

[jira] [Resolved] (SPARK-12351) Add documentation of submitting Mesos jobs with cluster mode

[ https://issues.apache.org/jira/browse/SPARK-12351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12351. --- Resolution: Fixed Assignee: Timothy Chen Fix Version/s: 2.0.0

[jira] [Resolved] (SPARK-9026) SimpleFutureAction.onComplete should not tie up a separate thread for each callback

[ https://issues.apache.org/jira/browse/SPARK-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-9026. -- Resolution: Fixed Fix Version/s: 2.0.0 > SimpleFutureAction.onComplete should not tie up a

[jira] [Resolved] (SPARK-10477) using DSL in ColumnPruningSuite to improve readablity

[ https://issues.apache.org/jira/browse/SPARK-10477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-10477. --- Resolution: Fixed Assignee: Wenchen Fan Fix Version/s: 2.0.0

[jira] [Resolved] (SPARK-12062) Master rebuilding historical SparkUI should be asynchronous

[ https://issues.apache.org/jira/browse/SPARK-12062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12062. --- Resolution: Fixed Fix Version/s: 2.0.0 1.6.1 > Master rebuilding

[jira] [Resolved] (SPARK-9516) Improve Thread Dump page

[ https://issues.apache.org/jira/browse/SPARK-9516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-9516. -- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > Improve Thread Dump

[jira] [Resolved] (SPARK-10123) Cannot set "--deploy-mode" in default configuration

[ https://issues.apache.org/jira/browse/SPARK-10123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-10123. --- Resolution: Fixed Assignee: Saisai Shao Fix Version/s: 2.0.0 > Cannot set

[jira] [Updated] (SPARK-9886) Validate usages of Runtime.getRuntime.addShutdownHook

[ https://issues.apache.org/jira/browse/SPARK-9886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-9886: - Assignee: Naveen Kumar Minchu > Validate usages of Runtime.getRuntime.addShutdownHook >

[jira] [Reopened] (SPARK-12062) Master rebuilding historical SparkUI should be asynchronous

[ https://issues.apache.org/jira/browse/SPARK-12062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reopened SPARK-12062: --- > Master rebuilding historical SparkUI should be asynchronous >

[jira] [Updated] (SPARK-12062) Master rebuilding historical SparkUI should be asynchronous

[ https://issues.apache.org/jira/browse/SPARK-12062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12062: -- Target Version/s: 1.6.1, 2.0.0 > Master rebuilding historical SparkUI should be asynchronous >

[jira] [Resolved] (SPARK-12275) No plan for BroadcastHint in some condition

[ https://issues.apache.org/jira/browse/SPARK-12275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12275. --- Resolution: Fixed Fix Version/s: 1.5.3 Target Version/s: 1.5.3, 1.6.1, 2.0.0 (was:

[jira] [Commented] (SPARK-12062) Master rebuilding historical SparkUI should be asynchronous

[ https://issues.apache.org/jira/browse/SPARK-12062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054829#comment-15054829 ] Andrew Or commented on SPARK-12062: --- I see, if you already have a patch then this is worth fixing.

[jira] [Commented] (SPARK-6270) Standalone Master hangs when streaming job completes and event logging is enabled

[ https://issues.apache.org/jira/browse/SPARK-6270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15053915#comment-15053915 ] Andrew Or commented on SPARK-6270: -- +1 to removing history serving functionality from standalone Master

[jira] [Commented] (SPARK-6270) Standalone Master hangs when streaming job completes and event logging is enabled

[ https://issues.apache.org/jira/browse/SPARK-6270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15053929#comment-15053929 ] Andrew Or commented on SPARK-6270: -- [~shivaram] That may be difficult to do because different

[jira] [Commented] (SPARK-12062) Master rebuilding historical SparkUI should be asynchronous

[ https://issues.apache.org/jira/browse/SPARK-12062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15053923#comment-15053923 ] Andrew Or commented on SPARK-12062: --- Actually, I'm closing this as a Won't Fix since SPARK-12299

[jira] [Created] (SPARK-12299) Remove history serving functionality from standalone Master

Andrew Or created SPARK-12299: - Summary: Remove history serving functionality from standalone Master Key: SPARK-12299 URL: https://issues.apache.org/jira/browse/SPARK-12299 Project: Spark Issue

[jira] [Resolved] (SPARK-12062) Master rebuilding historical SparkUI should be asynchronous

[ https://issues.apache.org/jira/browse/SPARK-12062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12062. --- Resolution: Won't Fix > Master rebuilding historical SparkUI should be asynchronous >

[jira] [Updated] (SPARK-4036) Add Conditional Random Fields (CRF) algorithm to Spark MLlib

[ https://issues.apache.org/jira/browse/SPARK-4036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Philpot updated SPARK-4036: -- Attachment: sample-output sample-input features.hair-eye

[jira] [Commented] (SPARK-6270) Standalone Master hangs when streaming job completes and event logging is enabled

[ https://issues.apache.org/jira/browse/SPARK-6270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15053395#comment-15053395 ] Andrew Or commented on SPARK-6270: -- Yeah, some regression tests would be helpful, though in my case I

[jira] [Commented] (SPARK-6270) Standalone Master hangs when streaming job completes and event logging is enabled

[ https://issues.apache.org/jira/browse/SPARK-6270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15053399#comment-15053399 ] Andrew Or commented on SPARK-6270: -- Also, we should consider switching event log compression on by

[jira] [Commented] (SPARK-6270) Standalone Master hangs when streaming job completes and event logging is enabled

[ https://issues.apache.org/jira/browse/SPARK-6270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15053405#comment-15053405 ] Andrew Or commented on SPARK-6270: -- You can set `spark.eventLog.compress` to true > Standalone Master

[jira] [Commented] (SPARK-6270) Standalone Master hangs when streaming job completes and event logging is enabled

[ https://issues.apache.org/jira/browse/SPARK-6270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15053924#comment-15053924 ] Andrew Or commented on SPARK-6270: -- I have filed a JIRA for it:

[jira] [Resolved] (SPARK-12108) Event logs are much bigger in 1.6 than in 1.5

[ https://issues.apache.org/jira/browse/SPARK-12108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12108. --- Resolution: Fixed Fix Version/s: 1.6.0 > Event logs are much bigger in 1.6 than in 1.5 >

[jira] [Resolved] (SPARK-12251) Document Spark 1.6's off-heap memory configurations and add config validation

[ https://issues.apache.org/jira/browse/SPARK-12251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12251. --- Resolution: Fixed > Document Spark 1.6's off-heap memory configurations and add config validation >

[jira] [Updated] (SPARK-12251) Document Spark 1.6's off-heap memory configurations and add config validation

[ https://issues.apache.org/jira/browse/SPARK-12251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12251: -- Fix Version/s: 1.6.0 > Document Spark 1.6's off-heap memory configurations and add config validation >

[jira] [Assigned] (SPARK-12155) Execution OOM after a relative large dataset cached in the cluster.

[ https://issues.apache.org/jira/browse/SPARK-12155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reassigned SPARK-12155: - Assignee: Andrew Or (was: Josh Rosen) > Execution OOM after a relative large dataset cached in

[jira] [Resolved] (SPARK-12155) Execution OOM after a relative large dataset cached in the cluster.

[ https://issues.apache.org/jira/browse/SPARK-12155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12155. --- Resolution: Fixed Fix Version/s: 1.6.0 > Execution OOM after a relative large dataset cached

[jira] [Resolved] (SPARK-12253) UnifiedMemoryManager race condition: storage can starve new tasks

[ https://issues.apache.org/jira/browse/SPARK-12253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12253. --- Resolution: Fixed Fix Version/s: 1.6.0 > UnifiedMemoryManager race condition: storage can

[jira] [Resolved] (SPARK-12189) UnifiedMemoryManager double counts storage memory freed

[ https://issues.apache.org/jira/browse/SPARK-12189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12189. --- Resolution: Fixed Fix Version/s: 1.6.0 > UnifiedMemoryManager double counts storage memory

[jira] [Created] (SPARK-12253) UnifiedMemoryManager race condition: storage can starve new tasks

Andrew Or created SPARK-12253: - Summary: UnifiedMemoryManager race condition: storage can starve new tasks Key: SPARK-12253 URL: https://issues.apache.org/jira/browse/SPARK-12253 Project: Spark

[jira] [Resolved] (SPARK-12165) Execution memory requests may fail to evict storage blocks if storage memory usage is below max memory

[ https://issues.apache.org/jira/browse/SPARK-12165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12165. --- Resolution: Fixed Fix Version/s: 1.6.0 > Execution memory requests may fail to evict storage

[jira] [Created] (SPARK-12205) Pivot fails Analysis when aggregate is UnresolvedFunction

Andrew Ray created SPARK-12205: -- Summary: Pivot fails Analysis when aggregate is UnresolvedFunction Key: SPARK-12205 URL: https://issues.apache.org/jira/browse/SPARK-12205 Project: Spark Issue

[jira] [Commented] (SPARK-10872) Derby error (XSDB6) when creating new HiveContext after restarting SparkContext

[ https://issues.apache.org/jira/browse/SPARK-10872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15047280#comment-15047280 ] Andrew King commented on SPARK-10872: - I am running into the same issue. I am calling HiveContext

[jira] [Created] (SPARK-12211) Incorrect version number in graphx doc for migration from 1.1

Andrew Ray created SPARK-12211: -- Summary: Incorrect version number in graphx doc for migration from 1.1 Key: SPARK-12211 URL: https://issues.apache.org/jira/browse/SPARK-12211 Project: Spark

[jira] [Created] (SPARK-12189) UnifiedMemoryManager double counts storage memory freed

Andrew Or created SPARK-12189: - Summary: UnifiedMemoryManager double counts storage memory freed Key: SPARK-12189 URL: https://issues.apache.org/jira/browse/SPARK-12189 Project: Spark Issue

[jira] [Created] (SPARK-12184) Make python api doc for pivot consistant with scala doc

Andrew Ray created SPARK-12184: -- Summary: Make python api doc for pivot consistant with scala doc Key: SPARK-12184 URL: https://issues.apache.org/jira/browse/SPARK-12184 Project: Spark Issue

[jira] [Commented] (SPARK-4036) Add Conditional Random Fields (CRF) algorithm to Spark MLlib

[ https://issues.apache.org/jira/browse/SPARK-4036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042462#comment-15042462 ] Andrew Philpot commented on SPARK-4036: --- Hi, what is the maturity of this code? Are you interested

[jira] [Commented] (SPARK-11081) Make spark-core pull in Jersey and javax.ws.rs dependencies separately for easier overriding

[ https://issues.apache.org/jira/browse/SPARK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042048#comment-15042048 ] Andrew Ash commented on SPARK-11081: Agreed, Spark 2.0 is the main opportunity to bump as many

[jira] [Created] (SPARK-12133) Support dynamic allocation in Spark Streaming

Andrew Or created SPARK-12133: - Summary: Support dynamic allocation in Spark Streaming Key: SPARK-12133 URL: https://issues.apache.org/jira/browse/SPARK-12133 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-12059) Standalone Master assertion error

[ https://issues.apache.org/jira/browse/SPARK-12059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12059. --- > Standalone Master assertion error > - > > Key:

[jira] [Updated] (SPARK-12110) spark-1.5.1-bin-hadoop2.6; pyspark.ml.feature Exception: ("You must build Spark with Hive

[ https://issues.apache.org/jira/browse/SPARK-12110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Davidson updated SPARK-12110: Attachment: launchCluster.sh.out launchCluster.sh launchCluster.sh is a

[jira] [Commented] (SPARK-12110) spark-1.5.1-bin-hadoop2.6; pyspark.ml.feature Exception: ("You must build Spark with Hive

[ https://issues.apache.org/jira/browse/SPARK-12110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15038574#comment-15038574 ] Andrew Davidson commented on SPARK-12110: - Hi Davies attached is a script I wrote to launch the

[jira] [Updated] (SPARK-12133) Support dynamic allocation in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12133: -- Target Version/s: 2.0.0 > Support dynamic allocation in Spark Streaming >

[jira] [Commented] (SPARK-12111) need upgrade instruction

[ https://issues.apache.org/jira/browse/SPARK-12111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15038563#comment-15038563 ] Andrew Davidson commented on SPARK-12111: - Hi Sean I am unable to find instructions for

[jira] [Updated] (SPARK-12110) spark-1.5.1-bin-hadoop2.6; pyspark.ml.feature Exception: ("You must build Spark with Hive

[ https://issues.apache.org/jira/browse/SPARK-12110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Davidson updated SPARK-12110: Attachment: launchingSparkCluster.md > spark-1.5.1-bin-hadoop2.6; pyspark.ml.feature

[jira] [Updated] (SPARK-12133) Support dynamic allocation in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12133: -- Attachment: dynamic-allocation-streaming-design.pdf > Support dynamic allocation in Spark Streaming >

[jira] [Updated] (SPARK-12133) Support dynamic allocation in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12133: -- Description: Dynamic allocation is a feature that allows your cluster resources to scale up and down

[jira] [Updated] (SPARK-12133) Support dynamic allocation in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12133: -- Assignee: Tathagata Das > Support dynamic allocation in Spark Streaming >

[jira] [Updated] (SPARK-12133) Support dynamic allocation in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12133: -- Description: Dynamic allocation is a feature that allows your cluster resources to scale up and down

[jira] [Created] (SPARK-12100) bug in spark/python/pyspark/rdd.py portable_hash()

Andrew Davidson created SPARK-12100: --- Summary: bug in spark/python/pyspark/rdd.py portable_hash() Key: SPARK-12100 URL: https://issues.apache.org/jira/browse/SPARK-12100 Project: Spark

[jira] [Created] (SPARK-12110) spark-1.5.1-bin-hadoop2.6; pyspark.ml.feature Exception: ("You must build Spark with Hive

Andrew Davidson created SPARK-12110: --- Summary: spark-1.5.1-bin-hadoop2.6; pyspark.ml.feature Exception: ("You must build Spark with Hive Key: SPARK-12110 URL:

[jira] [Commented] (SPARK-12111) need upgrade instruction

[ https://issues.apache.org/jira/browse/SPARK-12111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15036888#comment-15036888 ] Andrew Davidson commented on SPARK-12111: - This is where someone that knows the details of how

[jira] [Commented] (SPARK-12111) need upgrade instruction

[ https://issues.apache.org/jira/browse/SPARK-12111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15036887#comment-15036887 ] Andrew Davidson commented on SPARK-12111: - Hi Sean I understand I will need to stop by cluster

[jira] [Reopened] (SPARK-12111) need upgrade instruction

[ https://issues.apache.org/jira/browse/SPARK-12111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Davidson reopened SPARK-12111: - Hi Sean It must be possible for customers to upgrade installations. Given Spark is written

[jira] [Created] (SPARK-12108) Event logs are much bigger in 1.6 than in 1.5

Andrew Or created SPARK-12108: - Summary: Event logs are much bigger in 1.6 than in 1.5 Key: SPARK-12108 URL: https://issues.apache.org/jira/browse/SPARK-12108 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-12111) need upgrade instruction

Andrew Davidson created SPARK-12111: --- Summary: need upgrade instruction Key: SPARK-12111 URL: https://issues.apache.org/jira/browse/SPARK-12111 Project: Spark Issue Type: Documentation

[jira] [Commented] (SPARK-12110) spark-1.5.1-bin-hadoop2.6; pyspark.ml.feature Exception: ("You must build Spark with Hive

[ https://issues.apache.org/jira/browse/SPARK-12110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15037017#comment-15037017 ] Andrew Davidson commented on SPARK-12110: - Hi Patrick Here is how I start my notebook on my

[jira] [Commented] (SPARK-12110) spark-1.5.1-bin-hadoop2.6; pyspark.ml.feature Exception: ("You must build Spark with Hive

[ https://issues.apache.org/jira/browse/SPARK-12110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15037022#comment-15037022 ] Andrew Davidson commented on SPARK-12110: - Hi Patrick when I run the same example code on my

[jira] [Created] (SPARK-12081) Make unified memory management work with small heaps

Andrew Or created SPARK-12081: - Summary: Make unified memory management work with small heaps Key: SPARK-12081 URL: https://issues.apache.org/jira/browse/SPARK-12081 Project: Spark Issue Type:

[jira] [Commented] (SPARK-12059) Standalone Master assertion error

[ https://issues.apache.org/jira/browse/SPARK-12059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15035060#comment-15035060 ] Andrew Or commented on SPARK-12059: --- I think this only happens under error conditions, but it's still

[jira] [Resolved] (SPARK-12081) Make unified memory management work with small heaps

[ https://issues.apache.org/jira/browse/SPARK-12081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12081. --- Resolution: Fixed Fix Version/s: 1.6.0 > Make unified memory management work with small heaps

[jira] [Commented] (SPARK-12081) Make unified memory management work with small heaps

[ https://issues.apache.org/jira/browse/SPARK-12081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15035219#comment-15035219 ] Andrew Or commented on SPARK-12081: --- The patch took approach (1) > Make unified memory management work

[jira] [Resolved] (SPARK-5106) Make Web UI automatically refresh/update displayed data

[ https://issues.apache.org/jira/browse/SPARK-5106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-5106. -- Resolution: Won't Fix > Make Web UI automatically refresh/update displayed data >

[jira] [Commented] (SPARK-12062) Master rebuilding historical SparkUI should be asynchronous

[ https://issues.apache.org/jira/browse/SPARK-12062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15034405#comment-15034405 ] Andrew Or commented on SPARK-12062: --- Great, I've assigned it to you > Master rebuilding historical

<    5   6   7   8   9   10   11   12   13   14   >