[jira] [Comment Edited] (SPARK-3499) Create Spark-based distcp utility

[ https://issues.apache.org/jira/browse/SPARK-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14135842#comment-14135842 ] Patrick Wendell edited comment on SPARK-3499 at 9/16/14 6:03 PM

[jira] [Updated] (SPARK-787) Add EC2 Script Option to Push EC2 Credentials to Spark Nodes

[ https://issues.apache.org/jira/browse/SPARK-787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-787: -- Assignee: Dan Osipov (was: Patrick Cogan) Add EC2 Script Option to Push EC2 Credentials

[jira] [Resolved] (SPARK-787) Add EC2 Script Option to Push EC2 Credentials to Spark Nodes

[ https://issues.apache.org/jira/browse/SPARK-787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-787. --- Resolution: Fixed Fix Version/s: 1.2.0 This is fixed in https://github.com/apache/spark

[jira] [Created] (SPARK-3555) UI port contention suite flakey

Patrick Wendell created SPARK-3555: -- Summary: UI port contention suite flakey Key: SPARK-3555 URL: https://issues.apache.org/jira/browse/SPARK-3555 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-3555) UI port contention suite flakey

[ https://issues.apache.org/jira/browse/SPARK-3555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3555. Resolution: Fixed Fix Version/s: 1.1.1 1.2.0 Issue resolved

[jira] [Updated] (SPARK-3129) Prevent data loss in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3129: --- Component/s: Streaming Prevent data loss in Spark Streaming

[jira] [Resolved] (SPARK-3490) Alleviate port collisions during tests

[ https://issues.apache.org/jira/browse/SPARK-3490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3490. Resolution: Fixed Fix Version/s: 1.1.1 Issue resolved by pull request 2415 [https

[jira] [Updated] (SPARK-3489) support rdd.zip(rdd1, rdd2,...) with variable number of rdds as params

[ https://issues.apache.org/jira/browse/SPARK-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3489: --- Fix Version/s: (was: 1.0.3) support rdd.zip(rdd1, rdd2,...) with variable number of rdds

[jira] [Updated] (SPARK-3489) support rdd.zip(rdd1, rdd2,...) with variable number of rdds as params

[ https://issues.apache.org/jira/browse/SPARK-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3489: --- Target Version/s: 1.2.0 (was: 1.0.2) support rdd.zip(rdd1, rdd2,...) with variable number

[jira] [Commented] (SPARK-3561) Native Hadoop/YARN integration for batch/ETL workloads

[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14136777#comment-14136777 ] Patrick Wendell commented on SPARK-3561: Hey [~ozhurakousky] - could you provide

Re: greeting from new member and jira 3489

Hi Mohit, Welcome to the Spark community! We normally look at feature proposals using github pull requests mind submitting one? The contribution process is covered here: https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark On Tue, Sep 16, 2014 at 9:16 PM, Mohit Jaggi

Re: partitioned groupBy

If each partition can fit in memory, you can do this using mapPartitions and then building an inverse mapping within each partition. You'd need to construct a hash map within each partition yourself. On Tue, Sep 16, 2014 at 4:27 PM, Akshat Aranya aara...@gmail.com wrote: I have a use case where

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134198#comment-14134198 ] Patrick Wendell commented on SPARK-1239: Yes, the current state of the art

[jira] [Resolved] (SPARK-3425) OpenJDK - when run with jvm 1.8, should not set MaxPermSize

[ https://issues.apache.org/jira/browse/SPARK-3425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3425. Resolution: Fixed Fixed by: https://github.com/apache/spark/pull/2301 OpenJDK - when run

[jira] [Resolved] (SPARK-3518) Remove useless statement in JsonProtocol

[ https://issues.apache.org/jira/browse/SPARK-3518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3518. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Assignee

[jira] [Updated] (SPARK-2505) Weighted Regularizer

[ https://issues.apache.org/jira/browse/SPARK-2505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2505: --- Fix Version/s: (was: 1.1.0) 1.2.0 Weighted Regularizer

[jira] [Updated] (SPARK-2314) RDD actions are only overridden in Scala, not java or python

[ https://issues.apache.org/jira/browse/SPARK-2314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2314: --- Fix Version/s: (was: 1.1.0) 1.2.0 RDD actions are only overridden

[jira] [Updated] (SPARK-3403) NaiveBayes crashes with blas/lapack native libraries for breeze (netlib-java)

[ https://issues.apache.org/jira/browse/SPARK-3403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3403: --- Fix Version/s: (was: 1.1.0) 1.2.0 NaiveBayes crashes with blas/lapack

[jira] [Updated] (SPARK-2703) Make Tachyon related unit tests execute without deploying a Tachyon system locally.

[ https://issues.apache.org/jira/browse/SPARK-2703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2703: --- Fix Version/s: (was: 1.1.0) 1.2.0 Make Tachyon related unit tests

[jira] [Updated] (SPARK-3038) delete history server logs when there are too many logs

[ https://issues.apache.org/jira/browse/SPARK-3038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3038: --- Fix Version/s: (was: 1.1.0) 1.2.0 delete history server logs when

[jira] [Updated] (SPARK-1832) Executor UI improvement suggestions

[ https://issues.apache.org/jira/browse/SPARK-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1832: --- Fix Version/s: (was: 1.1.0) 1.2.0 Executor UI improvement suggestions

[jira] [Updated] (SPARK-2167) spark-submit should return exit code based on failure/success

[ https://issues.apache.org/jira/browse/SPARK-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2167: --- Fix Version/s: (was: 1.1.0) 1.2.0 spark-submit should return exit

[jira] [Updated] (SPARK-2754) Document standalone-cluster mode now that it's working

[ https://issues.apache.org/jira/browse/SPARK-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2754: --- Fix Version/s: (was: 1.1.0) 1.2.0 Document standalone-cluster mode

[jira] [Updated] (SPARK-2947) DAGScheduler resubmit the stage into an infinite loop

[ https://issues.apache.org/jira/browse/SPARK-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2947: --- Fix Version/s: (was: 1.1.0) 1.2.0 DAGScheduler resubmit the stage

[jira] [Updated] (SPARK-2638) Improve concurrency of fetching Map outputs

[ https://issues.apache.org/jira/browse/SPARK-2638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2638: --- Fix Version/s: (was: 1.1.0) 1.2.0 Improve concurrency of fetching Map

[jira] [Updated] (SPARK-1911) Warn users if their assembly jars are not built with Java 6

[ https://issues.apache.org/jira/browse/SPARK-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1911: --- Fix Version/s: (was: 1.1.0) 1.2.0 Warn users if their assembly jars

[jira] [Updated] (SPARK-2793) Correctly lock directory creation in DiskBlockManager.getFile

[ https://issues.apache.org/jira/browse/SPARK-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2793: --- Fix Version/s: (was: 1.1.0) 1.2.0 Correctly lock directory creation

[jira] [Updated] (SPARK-2069) MIMA false positives (umbrella)

[ https://issues.apache.org/jira/browse/SPARK-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2069: --- Fix Version/s: (was: 1.1.0) 1.2.0 MIMA false positives (umbrella

[jira] [Updated] (SPARK-1830) Deploy failover, Make Persistence engine and LeaderAgent Pluggable.

[ https://issues.apache.org/jira/browse/SPARK-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1830: --- Fix Version/s: (was: 1.1.0) 1.2.0 Deploy failover, Make Persistence

[jira] [Updated] (SPARK-2795) Improve DiskBlockObjectWriter API

[ https://issues.apache.org/jira/browse/SPARK-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2795: --- Fix Version/s: (was: 1.1.0) 1.2.0 Improve DiskBlockObjectWriter API

[jira] [Updated] (SPARK-1706) Allow multiple executors per worker in Standalone mode

[ https://issues.apache.org/jira/browse/SPARK-1706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1706: --- Fix Version/s: (was: 1.1.0) 1.2.0 Allow multiple executors per worker

[jira] [Updated] (SPARK-1989) Exit executors faster if they get into a cycle of heavy GC

[ https://issues.apache.org/jira/browse/SPARK-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1989: --- Fix Version/s: (was: 1.1.0) 1.2.0 Exit executors faster if they get

[jira] [Updated] (SPARK-1860) Standalone Worker cleanup should not clean up running executors

[ https://issues.apache.org/jira/browse/SPARK-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1860: --- Fix Version/s: (was: 1.1.0) 1.2.0 Standalone Worker cleanup should

[jira] [Updated] (SPARK-1924) Make local:/ scheme work in more deploy modes

[ https://issues.apache.org/jira/browse/SPARK-1924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1924: --- Fix Version/s: (was: 1.1.0) 1.2.0 Make local:/ scheme work in more

[jira] [Updated] (SPARK-1379) Calling .cache() on a SchemaRDD should do something more efficient than caching the individual row objects.

[ https://issues.apache.org/jira/browse/SPARK-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1379: --- Fix Version/s: (was: 1.1.0) 1.2.0 Calling .cache() on a SchemaRDD

[jira] [Updated] (SPARK-1684) Merge script should standardize SPARK-XXX prefix

[ https://issues.apache.org/jira/browse/SPARK-1684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1684: --- Fix Version/s: (was: 1.1.0) 1.2.0 Merge script should standardize

[jira] [Updated] (SPARK-1853) Show Streaming application code context (file, line number) in Spark Stages UI

[ https://issues.apache.org/jira/browse/SPARK-1853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1853: --- Fix Version/s: (was: 1.1.0) 1.2.0 Show Streaming application code

[jira] [Updated] (SPARK-1201) Do not materialize partitions whenever possible in BlockManager

[ https://issues.apache.org/jira/browse/SPARK-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1201: --- Fix Version/s: (was: 1.1.0) 1.2.0 Do not materialize partitions

[jira] [Updated] (SPARK-2159) Spark shell exit() does not stop SparkContext

[ https://issues.apache.org/jira/browse/SPARK-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2159: --- Fix Version/s: (was: 1.1.0) 1.2.0 Spark shell exit() does not stop

[jira] [Updated] (SPARK-1792) Missing Spark-Shell Configure Options

[ https://issues.apache.org/jira/browse/SPARK-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1792: --- Fix Version/s: (was: 1.1.0) 1.2.0 Missing Spark-Shell Configure

[jira] [Updated] (SPARK-1412) Disable partial aggregation automatically when reduction factor is low

[ https://issues.apache.org/jira/browse/SPARK-1412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1412: --- Fix Version/s: (was: 1.1.0) 1.2.0 Disable partial aggregation

[jira] [Updated] (SPARK-2584) Do not mutate block storage level on the UI

[ https://issues.apache.org/jira/browse/SPARK-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2584: --- Fix Version/s: (was: 1.1.0) 1.2.0 Do not mutate block storage level

[jira] [Updated] (SPARK-2624) Datanucleus jars not accessible in yarn-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-2624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2624: --- Fix Version/s: (was: 1.1.0) 1.2.0 Datanucleus jars not accessible

[jira] [Updated] (SPARK-1127) Add saveAsHBase to PairRDDFunctions

[ https://issues.apache.org/jira/browse/SPARK-1127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1127: --- Fix Version/s: (was: 1.1.0) 1.2.0 Add saveAsHBase to PairRDDFunctions

[jira] [Updated] (SPARK-1338) Create Additional Style Rules

[ https://issues.apache.org/jira/browse/SPARK-1338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1338: --- Fix Version/s: (was: 1.1.0) 1.2.0 Create Additional Style Rules

[jira] [Updated] (SPARK-3505) Augmenting SparkStreaming updateStateByKey API with timestamp

[ https://issues.apache.org/jira/browse/SPARK-3505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3505: --- Fix Version/s: (was: 1.1.0) 1.2.0 Augmenting SparkStreaming

[jira] [Updated] (SPARK-1972) Add support for setting and visualizing custom task-related metrics

[ https://issues.apache.org/jira/browse/SPARK-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1972: --- Fix Version/s: (was: 1.1.0) 1.2.0 Add support for setting

[jira] [Updated] (SPARK-2913) Spark's log4j.properties should always appear ahead of Hadoop's on classpath

[ https://issues.apache.org/jira/browse/SPARK-2913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2913: --- Fix Version/s: (was: 1.1.0) 1.2.0 Spark's log4j.properties should

[jira] [Updated] (SPARK-2794) Use Java 7 isSymlink when available

[ https://issues.apache.org/jira/browse/SPARK-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2794: --- Fix Version/s: (was: 1.1.0) 1.2.0 Use Java 7 isSymlink when available

[jira] [Updated] (SPARK-1761) Add broadcast information on SparkUI storage tab

[ https://issues.apache.org/jira/browse/SPARK-1761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1761: --- Fix Version/s: (was: 1.1.0) 1.2.0 Add broadcast information

[jira] [Updated] (SPARK-1627) Support external aggregation in Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-1627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1627: --- Fix Version/s: (was: 1.1.0) 1.2.0 Support external aggregation

[jira] [Updated] (SPARK-1921) Allow duplicate jar files among the app jar and secondary jars in yarn-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-1921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1921: --- Fix Version/s: (was: 1.1.0) 1.2.0 Allow duplicate jar files among

[jira] [Updated] (SPARK-1529) Support setting spark.local.dirs to a hadoop FileSystem

[ https://issues.apache.org/jira/browse/SPARK-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1529: --- Fix Version/s: (was: 1.1.0) 1.2.0 Support setting spark.local.dirs

[jira] [Updated] (SPARK-2722) Mechanism for escaping spark configs is not consistent

[ https://issues.apache.org/jira/browse/SPARK-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2722: --- Fix Version/s: (was: 1.1.0) 1.2.0 Mechanism for escaping spark

[jira] [Updated] (SPARK-2220) Fix remaining Hive Commands

[ https://issues.apache.org/jira/browse/SPARK-2220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2220: --- Fix Version/s: (was: 1.1.0) 1.2.0 Fix remaining Hive Commands

[jira] [Updated] (SPARK-2068) Remove other uses of @transient lazy val in physical plan nodes

[ https://issues.apache.org/jira/browse/SPARK-2068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2068: --- Fix Version/s: (was: 1.1.0) 1.2.0 Remove other uses of @transient

[jira] [Updated] (SPARK-1866) Closure cleaner does not null shadowed fields when outer scope is referenced

[ https://issues.apache.org/jira/browse/SPARK-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1866: --- Fix Version/s: (was: 1.1.0) 1.2.0 Closure cleaner does not null

[jira] [Updated] (SPARK-3191) Add explanation of supporting building spark with maven in http proxy environment

[ https://issues.apache.org/jira/browse/SPARK-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3191: --- Fix Version/s: (was: 1.1.0) 1.2.0 Add explanation of supporting

[jira] [Updated] (SPARK-2626) Stop SparkContext in all examples

[ https://issues.apache.org/jira/browse/SPARK-2626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2626: --- Fix Version/s: (was: 1.1.0) 1.2.0 Stop SparkContext in all examples

[jira] [Updated] (SPARK-2770) Rename spark-ganglia-lgpl to ganglia-lgpl

[ https://issues.apache.org/jira/browse/SPARK-2770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2770: --- Fix Version/s: (was: 1.1.0) 1.2.0 Rename spark-ganglia-lgpl

[jira] [Updated] (SPARK-2914) spark.*.extraJavaOptions are evaluated too many times

[ https://issues.apache.org/jira/browse/SPARK-2914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2914: --- Fix Version/s: (was: 1.1.0) 1.2.0 spark.*.extraJavaOptions

[jira] [Updated] (SPARK-2672) support compressed file in wholeFile()

[ https://issues.apache.org/jira/browse/SPARK-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2672: --- Fix Version/s: (was: 1.1.0) 1.2.0 support compressed file

[jira] [Updated] (SPARK-1389) Make numPartitions in Exchange configurable

[ https://issues.apache.org/jira/browse/SPARK-1389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1389: --- Fix Version/s: (was: 1.1.0) 1.2.0 Make numPartitions in Exchange

[jira] [Updated] (SPARK-1455) Determine which test suites to run based on code changes

[ https://issues.apache.org/jira/browse/SPARK-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1455: --- Fix Version/s: (was: 1.1.0) 1.2.0 Determine which test suites to run

[jira] [Updated] (SPARK-1564) Add JavaScript into Javadoc to turn ::Experimental:: and such into badges

[ https://issues.apache.org/jira/browse/SPARK-1564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1564: --- Fix Version/s: (was: 1.1.0) 1.2.0 Add JavaScript into Javadoc to turn

[jira] [Updated] (SPARK-1719) spark.executor.extraLibraryPath isn't applied on yarn

[ https://issues.apache.org/jira/browse/SPARK-1719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1719: --- Fix Version/s: (was: 1.1.0) 1.2.0 spark.executor.extraLibraryPath

[jira] [Updated] (SPARK-2460) Optimize SparkContext.hadoopFile api

[ https://issues.apache.org/jira/browse/SPARK-2460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2460: --- Fix Version/s: (was: 1.1.0) 1.2.0 Optimize SparkContext.hadoopFile

[jira] [Updated] (SPARK-1369) HiveUDF wrappers are slow

[ https://issues.apache.org/jira/browse/SPARK-1369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1369: --- Fix Version/s: (was: 1.1.0) 1.2.0 HiveUDF wrappers are slow

[jira] [Updated] (SPARK-1694) Simplify ColumnBuilder/Accessor class hierarchy

[ https://issues.apache.org/jira/browse/SPARK-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1694: --- Fix Version/s: (was: 1.1.0) 1.2.0 Simplify ColumnBuilder/Accessor

[jira] [Updated] (SPARK-1714) Take advantage of AMRMClient APIs to simplify logic in YarnAllocationHandler

[ https://issues.apache.org/jira/browse/SPARK-1714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1714: --- Fix Version/s: (was: 1.1.0) 1.2.0 Take advantage of AMRMClient APIs

[jira] [Updated] (SPARK-1825) Windows Spark fails to work with Linux YARN

[ https://issues.apache.org/jira/browse/SPARK-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1825: --- Fix Version/s: (was: 1.1.0) 1.2.0 Windows Spark fails to work

[jira] [Updated] (SPARK-1172) Improve naming of the BlockManager classes

[ https://issues.apache.org/jira/browse/SPARK-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1172: --- Fix Version/s: (was: 1.1.0) 1.2.0 Improve naming of the BlockManager

[jira] [Updated] (SPARK-1739) Close PR's after 30 days of inactivity

[ https://issues.apache.org/jira/browse/SPARK-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1739: --- Fix Version/s: (was: 1.1.0) 1.2.0 Close PR's after 30 days

[jira] [Updated] (SPARK-922) Update Spark AMI to Python 2.7

[ https://issues.apache.org/jira/browse/SPARK-922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-922: -- Fix Version/s: (was: 1.1.0) 1.2.0 Update Spark AMI to Python 2.7

[jira] [Updated] (SPARK-1079) EC2 scripts should allow mounting as XFS or EXT4

[ https://issues.apache.org/jira/browse/SPARK-1079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1079: --- Fix Version/s: (was: 1.1.0) 1.2.0 EC2 scripts should allow mounting

[jira] [Updated] (SPARK-1762) Add functionality to pin RDDs in cache

[ https://issues.apache.org/jira/browse/SPARK-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1762: --- Fix Version/s: (was: 1.1.0) 1.2.0 Add functionality to pin RDDs

[jira] [Updated] (SPARK-1272) Don't fail job if some local directories are buggy

[ https://issues.apache.org/jira/browse/SPARK-1272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1272: --- Fix Version/s: (was: 1.1.0) 1.2.0 Don't fail job if some local

[jira] [Updated] (SPARK-1652) Fixes and improvements for spark-submit/configs

[ https://issues.apache.org/jira/browse/SPARK-1652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1652: --- Fix Version/s: (was: 1.1.0) 1.2.0 Fixes and improvements for spark

[jira] [Updated] (SPARK-1069) Provide binary compatibility in Spark 1.X releases

[ https://issues.apache.org/jira/browse/SPARK-1069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1069: --- Fix Version/s: (was: 1.1.0) 1.2.0 Provide binary compatibility

[jira] [Updated] (SPARK-874) Have a --wait flag in ./sbin/stop-all.sh that polls until Worker's are finished

[ https://issues.apache.org/jira/browse/SPARK-874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-874: -- Fix Version/s: (was: 1.1.0) 1.2.0 Have a --wait flag in ./sbin/stop

[jira] [Assigned] (SPARK-3506) 1.1.0-SNAPSHOT in docs for 1.1.0 under docs/latest

[ https://issues.apache.org/jira/browse/SPARK-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell reassigned SPARK-3506: -- Assignee: Patrick Wendell 1.1.0-SNAPSHOT in docs for 1.1.0 under docs/latest

[jira] [Commented] (SPARK-3506) 1.1.0-SNAPSHOT in docs for 1.1.0 under docs/latest

[ https://issues.apache.org/jira/browse/SPARK-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134690#comment-14134690 ] Patrick Wendell commented on SPARK-3506: Yeah the issue is some people wanted

[jira] [Created] (SPARK-3543) Write TaskContext in Java and expose it through a static accessor

Patrick Wendell created SPARK-3543: -- Summary: Write TaskContext in Java and expose it through a static accessor Key: SPARK-3543 URL: https://issues.apache.org/jira/browse/SPARK-3543 Project: Spark

[jira] [Updated] (SPARK-3543) Write TaskContext in Java and expose it through a static accessor

[ https://issues.apache.org/jira/browse/SPARK-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3543: --- Description: Right now we have these xWithContext methods and it's a bit awkward

[jira] [Updated] (SPARK-3543) Write TaskContext in Java and expose it through a static accessor

[ https://issues.apache.org/jira/browse/SPARK-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3543: --- Priority: Critical (was: Major) Write TaskContext in Java and expose it through a static

[jira] [Resolved] (SPARK-3040) Reverse order of Utils.findLocalIpAddress on UNIX

[ https://issues.apache.org/jira/browse/SPARK-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3040. Resolution: Fixed Fix Version/s: 1.2.0 Target Version/s: (was: 1.1.0

[jira] [Updated] (SPARK-3040) Reverse order of Utils.findLocalIpAddress on UNIX

[ https://issues.apache.org/jira/browse/SPARK-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3040: --- Assignee: Ye Xianjin Reverse order of Utils.findLocalIpAddress on UNIX

[jira] [Updated] (SPARK-3040) Reverse order of Utils.findLocalIpAddress on UNIX

[ https://issues.apache.org/jira/browse/SPARK-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3040: --- Summary: Reverse order of Utils.findLocalIpAddress on UNIX (was: pick up a more proper local

[jira] [Updated] (SPARK-1449) Please delete old releases from mirroring system

[ https://issues.apache.org/jira/browse/SPARK-1449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1449: --- Affects Version/s: (was: 1.0.2) Please delete old releases from mirroring system

[jira] [Updated] (SPARK-1449) Please delete old releases from mirroring system

[ https://issues.apache.org/jira/browse/SPARK-1449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1449: --- Fix Version/s: 0.8.1 0.9.1 1.0.0

[jira] [Resolved] (SPARK-1449) Please delete old releases from mirroring system

[ https://issues.apache.org/jira/browse/SPARK-1449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1449. Resolution: Fixed I've left 1.0.2 and 1.1.0, since 1.1.0 is an unstable release. Please

[jira] [Updated] (SPARK-3039) Spark assembly for new hadoop API (hadoop 2) contains avro-mapred for hadoop 1 API

[ https://issues.apache.org/jira/browse/SPARK-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3039: --- Assignee: Bertrand Bossy Spark assembly for new hadoop API (hadoop 2) contains avro-mapred

[jira] [Resolved] (SPARK-3039) Spark assembly for new hadoop API (hadoop 2) contains avro-mapred for hadoop 1 API

[ https://issues.apache.org/jira/browse/SPARK-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3039. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Target

[jira] [Resolved] (SPARK-3452) Maven build should skip publishing artifacts people shouldn't depend on

[ https://issues.apache.org/jira/browse/SPARK-3452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3452. Resolution: Fixed Fixed by: https://github.com/apache/spark/pull/2329 Maven build should

Tests and Test Infrastructure

Hey All, Wanted to send a quick update about test infrastructure. With the number of contributors we have and the rate of development, maintaining a well-oiled test infra is really important. Every time a flaky test fails a legitimate pull request, it wastes developer time and effort. 1. Master

Re: spark-1.1.0 with make-distribution.sh problem

Yeah that issue has been fixed by adding better docs, it just didn't make it in time for the release: https://github.com/apache/spark/blob/branch-1.1/make-distribution.sh#L54 On Thu, Sep 11, 2014 at 11:57 PM, Zhanfeng Huo huozhanf...@gmail.com wrote: resolved： ./make-distribution.sh --name

[jira] [Updated] (SPARK-3438) Support for accessing secured HDFS in Standalone Mode

[ https://issues.apache.org/jira/browse/SPARK-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3438: --- Description: Access to secured HDFS is currently supported in YARN using YARN's built

[jira] [Created] (SPARK-3511) Create a RELEASE-NOTES.txt file in the repo

Patrick Wendell created SPARK-3511: -- Summary: Create a RELEASE-NOTES.txt file in the repo Key: SPARK-3511 URL: https://issues.apache.org/jira/browse/SPARK-3511 Project: Spark Issue Type

[jira] [Commented] (SPARK-3500) SchemaRDD from jsonRDD() has not coalesce() method

[ https://issues.apache.org/jira/browse/SPARK-3500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131706#comment-14131706 ] Patrick Wendell commented on SPARK-3500: If it's just a missing feature we tend

[jira] [Updated] (SPARK-3513) Provide a utility for running a function once on each executor

[ https://issues.apache.org/jira/browse/SPARK-3513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3513: --- Fix Version/s: (was: 1.2.0) Provide a utility for running a function once on each

<    17   18   19   20   21   22   23   24   25   26   >