[jira] [Resolved] (SPARK-6055) Memory leak in pyspark sql due to incorrect equality check

[ https://issues.apache.org/jira/browse/SPARK-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-6055. --- Resolution: Fixed Fix Version/s: 1.2.2 1.1.2 1.3.0

[jira] [Commented] (SPARK-5950) Insert array into a metastore table saved as parquet should work when using datasource api

[ https://issues.apache.org/jira/browse/SPARK-5950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341361#comment-14341361 ] Apache Spark commented on SPARK-5950: - User 'yhuai' has created a pull request for

[jira] [Commented] (SPARK-6079) Use index to speed up StatusTracker.getJobIdsForGroup()

[ https://issues.apache.org/jira/browse/SPARK-6079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341405#comment-14341405 ] Apache Spark commented on SPARK-6079: - User 'JoshRosen' has created a pull request for

[jira] [Updated] (SPARK-6073) Need invalidate metastore cache after append data in CreateMetastoreDataSourceAsSelect

[ https://issues.apache.org/jira/browse/SPARK-6073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-6073: Summary: Need invalidate metastore cache after append data in CreateMetastoreDataSourceAsSelect (was: Need

[jira] [Created] (SPARK-6074) Assembly doesn't include pyspark sql files

Marcelo Vanzin created SPARK-6074: - Summary: Assembly doesn't include pyspark sql files Key: SPARK-6074 URL: https://issues.apache.org/jira/browse/SPARK-6074 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-6074) Assembly doesn't include pyspark sql files

[ https://issues.apache.org/jira/browse/SPARK-6074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341252#comment-14341252 ] Marcelo Vanzin commented on SPARK-6074: --- Testing a fix, will send PR shortly. I'm

[jira] [Commented] (SPARK-6074) Assembly doesn't include pyspark sql files

[ https://issues.apache.org/jira/browse/SPARK-6074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341262#comment-14341262 ] Apache Spark commented on SPARK-6074: - User 'vanzin' has created a pull request for

[jira] [Updated] (SPARK-6073) Need to refresh metastore cache after append data in CreateMetastoreDataSourceAsSelect

[ https://issues.apache.org/jira/browse/SPARK-6073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-6073: Summary: Need to refresh metastore cache after append data in CreateMetastoreDataSourceAsSelect (was: Need

[jira] [Commented] (SPARK-6073) Need to refresh metastore cache after append data in CreateMetastoreDataSourceAsSelect

[ https://issues.apache.org/jira/browse/SPARK-6073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341319#comment-14341319 ] Apache Spark commented on SPARK-6073: - User 'yhuai' has created a pull request for

[jira] [Commented] (SPARK-6078) create event log directory automatically if not exists

[ https://issues.apache.org/jira/browse/SPARK-6078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341393#comment-14341393 ] Apache Spark commented on SPARK-6078: - User 'liyezhang556520' has created a pull

[jira] [Resolved] (SPARK-5979) `--packages` should not exclude spark streaming assembly jars for kafka and flume

[ https://issues.apache.org/jira/browse/SPARK-5979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5979. Resolution: Fixed Fix Version/s: 1.3.0 `--packages` should not exclude spark

[jira] [Commented] (SPARK-6068) KMeans Parallel test may fail

[ https://issues.apache.org/jira/browse/SPARK-6068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341299#comment-14341299 ] Sean Owen commented on SPARK-6068: -- Has the test failed or is this theoretical? Fixing

[jira] [Commented] (SPARK-869) Retrofit rest of RDD api to use proper serializer type

[ https://issues.apache.org/jira/browse/SPARK-869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341300#comment-14341300 ] Sean Owen commented on SPARK-869: - Is this still live -- what are other methods that need

[jira] [Commented] (SPARK-6068) KMeans Parallel test may fail

[ https://issues.apache.org/jira/browse/SPARK-6068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341316#comment-14341316 ] Derrick Burns commented on SPARK-6068: -- Not theoretical. The unit test failed for me

[jira] [Resolved] (SPARK-6070) Yarn Shuffle Service jar packages too many dependencies

[ https://issues.apache.org/jira/browse/SPARK-6070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-6070. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Marcelo Vanzin Yarn

[jira] [Updated] (SPARK-5979) `--packages` should not exclude spark streaming assembly jars for kafka and flume

[ https://issues.apache.org/jira/browse/SPARK-5979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5979: --- Assignee: Burak Yavuz `--packages` should not exclude spark streaming assembly jars for

[jira] [Updated] (SPARK-6048) SparkConf.translateConfKey should not translate on set

[ https://issues.apache.org/jira/browse/SPARK-6048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-6048: - Summary: SparkConf.translateConfKey should not translate on set (was: SparkConf.translateConfKey should

[jira] [Commented] (SPARK-6056) Unlimit offHeap memory use cause RM killing the container

[ https://issues.apache.org/jira/browse/SPARK-6056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341260#comment-14341260 ] Lianhui Wang commented on SPARK-6056: - [~adav] from your given information, when

[jira] [Resolved] (SPARK-3070) Kryo deserialization without using the custom registrator

[ https://issues.apache.org/jira/browse/SPARK-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3070. -- Resolution: Duplicate Kryo deserialization without using the custom registrator

[jira] [Resolved] (SPARK-6042) spark-submit giving Exception in thread main java.lang.NoSuchMethodError: org.apache.spark.sql.hive.HiveContext.sql(Ljava/lang/String;)Lorg/apache/spark/sql/SchemaRDD;

[ https://issues.apache.org/jira/browse/SPARK-6042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6042. -- Resolution: Not a Problem We can reopen if there is evidence that this is not due to Spark version

[jira] [Updated] (SPARK-6064) Checking data types when resolving types

[ https://issues.apache.org/jira/browse/SPARK-6064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6064: - Component/s: SQL Checking data types when resolving types

[jira] [Commented] (SPARK-6068) KMeans Parallel test may fail

[ https://issues.apache.org/jira/browse/SPARK-6068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341326#comment-14341326 ] Sean Owen commented on SPARK-6068: -- Is this something for which a PR can easily be

[jira] [Commented] (SPARK-6056) Unlimit offHeap memory use cause RM killing the container

[ https://issues.apache.org/jira/browse/SPARK-6056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341344#comment-14341344 ] Aaron Davidson commented on SPARK-6056: --- This should already be the case:

[jira] [Created] (SPARK-6075) Flaky AccumulatorSuite.add value to collection accumulators test

Josh Rosen created SPARK-6075: - Summary: Flaky AccumulatorSuite.add value to collection accumulators test Key: SPARK-6075 URL: https://issues.apache.org/jira/browse/SPARK-6075 Project: Spark

[jira] [Updated] (SPARK-6078) create event log directory automatically if not exists

[ https://issues.apache.org/jira/browse/SPARK-6078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhang, Liye updated SPARK-6078: --- Description: when event log directory does not exists, spark just throw IlleagalArgumentException and

[jira] [Comment Edited] (SPARK-5791) [Spark SQL] show poor performance when multiple table do join operation

[ https://issues.apache.org/jira/browse/SPARK-5791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341095#comment-14341095 ] Yi Zhou edited comment on SPARK-5791 at 2/28/15 1:36 AM: - Add

[jira] [Updated] (SPARK-6073) Need invalidate metastore cache after append data in CreateMetastoreDataSourceAsSelect

[ https://issues.apache.org/jira/browse/SPARK-6073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-6073: Description: We should drop the metadata cache in CreateMetastoreDataSourceAsSelect after we append data.

[jira] [Commented] (SPARK-5945) Spark should not retry a stage infinitely on a FetchFailedException

[ https://issues.apache.org/jira/browse/SPARK-5945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341285#comment-14341285 ] SuYan commented on SPARK-5945: -- I encounter stage retry infinitely when a executor lost

[jira] [Resolved] (SPARK-6032) Move ivy logging to System.err in --packages

[ https://issues.apache.org/jira/browse/SPARK-6032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-6032. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Burak Yavuz Move ivy

[jira] [Created] (SPARK-6078) create event log directory automatically if not exists

Zhang, Liye created SPARK-6078: -- Summary: create event log directory automatically if not exists Key: SPARK-6078 URL: https://issues.apache.org/jira/browse/SPARK-6078 Project: Spark Issue Type:

[jira] [Commented] (SPARK-4411) Add kill link for jobs in the UI

[ https://issues.apache.org/jira/browse/SPARK-4411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341291#comment-14341291 ] Apache Spark commented on SPARK-4411: - User 'lianhuiwang' has created a pull request

[jira] [Created] (SPARK-6077) Multiple spark streaming tabs on UI when reuse the same sparkcontext

zhichao-li created SPARK-6077: - Summary: Multiple spark streaming tabs on UI when reuse the same sparkcontext Key: SPARK-6077 URL: https://issues.apache.org/jira/browse/SPARK-6077 Project: Spark

[jira] [Commented] (SPARK-6077) Multiple spark streaming tabs on UI when reuse the same sparkcontext

[ https://issues.apache.org/jira/browse/SPARK-6077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341388#comment-14341388 ] Apache Spark commented on SPARK-6077: - User 'zhichao-li' has created a pull request

[jira] [Updated] (SPARK-6073) Need refresh metastore cache after append data in CreateMetastoreDataSourceAsSelect

[ https://issues.apache.org/jira/browse/SPARK-6073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-6073: Summary: Need refresh metastore cache after append data in CreateMetastoreDataSourceAsSelect (was: Need

[jira] [Resolved] (SPARK-4739) spark.files.userClassPathFirst does not work in local[*] mode

[ https://issues.apache.org/jira/browse/SPARK-4739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4739. -- Resolution: Duplicate Duplicate on the grounds that, if there's a follow up, it seems like it's best

[jira] [Updated] (SPARK-6069) Deserialization Error ClassNotFound

[ https://issues.apache.org/jira/browse/SPARK-6069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6069: - Priority: Major (was: Blocker) Fix Version/s: (was: 1.2.2) This can be bumped up after some

[jira] [Commented] (SPARK-6068) KMeans Parallel test may fail

[ https://issues.apache.org/jira/browse/SPARK-6068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341292#comment-14341292 ] Derrick Burns commented on SPARK-6068: -- I've tried providing pull requests to Spark

[jira] [Created] (SPARK-6079) Use index to speed up StatusTracker.getJobIdsForGroup()

Josh Rosen created SPARK-6079: - Summary: Use index to speed up StatusTracker.getJobIdsForGroup() Key: SPARK-6079 URL: https://issues.apache.org/jira/browse/SPARK-6079 Project: Spark Issue Type:

[jira] [Created] (SPARK-6073) Need invalidate metastore cache after append data in CreatableRelationProvider.createRelation

Yin Huai created SPARK-6073: --- Summary: Need invalidate metastore cache after append data in CreatableRelationProvider.createRelation Key: SPARK-6073 URL: https://issues.apache.org/jira/browse/SPARK-6073

[jira] [Commented] (SPARK-5206) Accumulators are not re-registered during recovering from checkpoint

[ https://issues.apache.org/jira/browse/SPARK-5206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341343#comment-14341343 ] zzc commented on SPARK-5206: I have same problem, [~vincentye38], how to resolve this?

[jira] [Created] (SPARK-6076) Fix a potential OOM issue when StorageLevel is MEMORY_AND_DISK_SER

Shixiong Zhu created SPARK-6076: --- Summary: Fix a potential OOM issue when StorageLevel is MEMORY_AND_DISK_SER Key: SPARK-6076 URL: https://issues.apache.org/jira/browse/SPARK-6076 Project: Spark

[jira] [Commented] (SPARK-6076) Fix a potential OOM issue when StorageLevel is MEMORY_AND_DISK_SER

[ https://issues.apache.org/jira/browse/SPARK-6076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341386#comment-14341386 ] Apache Spark commented on SPARK-6076: - User 'zsxwing' has created a pull request for

[jira] [Resolved] (SPARK-6049) HiveThriftServer2 may expose Inheritable methods

[ https://issues.apache.org/jira/browse/SPARK-6049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Littlestar resolved SPARK-6049. --- Resolution: Invalid very sorry, HiveThriftServer2.startWithContext can be called by java. I will open

[jira] [Resolved] (SPARK-5434) Preserve spaces in path to spark-ec2

[ https://issues.apache.org/jira/browse/SPARK-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5434. -- Resolution: Fixed Fix Version/s: 1.2.2 Target Version/s: (was: 1.3.0, 1.2.1) I

[jira] [Reopened] (SPARK-4900) MLlib SingularValueDecomposition ARPACK IllegalStateException

[ https://issues.apache.org/jira/browse/SPARK-4900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Beyer reopened SPARK-4900: --- i traced the problem back due to Double.NaN, Double.POSITIVE_INFINITY or Double.NEGATIVE_INFINITY

[jira] [Commented] (SPARK-5654) Integrate SparkR into Apache Spark

[ https://issues.apache.org/jira/browse/SPARK-5654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340355#comment-14340355 ] Hari Sekhon commented on SPARK-5654: Ok replace the word packaging with upstream

[jira] [Comment Edited] (SPARK-5654) Integrate SparkR into Apache Spark

[ https://issues.apache.org/jira/browse/SPARK-5654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340355#comment-14340355 ] Hari Sekhon edited comment on SPARK-5654 at 2/27/15 4:42 PM: -

[jira] [Updated] (SPARK-5556) Latent Dirichlet Allocation (LDA) using Gibbs sampler

[ https://issues.apache.org/jira/browse/SPARK-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-5556: - Shepherd: Joseph K. Bradley Assignee: Pedro Rodriguez Latent Dirichlet Allocation (LDA)

[jira] [Updated] (SPARK-5556) Latent Dirichlet Allocation (LDA) using Gibbs sampler

[ https://issues.apache.org/jira/browse/SPARK-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-5556: - Target Version/s: 1.4.0 Latent Dirichlet Allocation (LDA) using Gibbs sampler

[jira] [Commented] (SPARK-6050) Spark on YARN does not work --executor-cores is specified

[ https://issues.apache.org/jira/browse/SPARK-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340411#comment-14340411 ] Marcelo Vanzin commented on SPARK-6050: --- Is there any downside to always requesting

[jira] [Commented] (SPARK-6050) Spark on YARN does not work --executor-cores is specified

[ https://issues.apache.org/jira/browse/SPARK-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340433#comment-14340433 ] Thomas Graves commented on SPARK-6050: -- Note, the problem is in determining whether

[jira] [Commented] (SPARK-6048) SparkConf.translateConfKey should translate on get, not set

[ https://issues.apache.org/jira/browse/SPARK-6048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340398#comment-14340398 ] Marcelo Vanzin commented on SPARK-6048: --- But the current patch has a precedence

[jira] [Commented] (SPARK-6050) Spark on YARN does not work --executor-cores is specified

[ https://issues.apache.org/jira/browse/SPARK-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340379#comment-14340379 ] Mridul Muralidharan commented on SPARK-6050: [~tgraves] You are right, cpu

[jira] [Commented] (SPARK-6050) Spark on YARN does not work --executor-cores is specified

[ https://issues.apache.org/jira/browse/SPARK-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340431#comment-14340431 ] Thomas Graves commented on SPARK-6050: -- Ah good point. Note that 1.2 didn't always

[jira] [Resolved] (SPARK-5809) OutOfMemoryError in logDebug in RandomForest.scala

[ https://issues.apache.org/jira/browse/SPARK-5809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5809. -- Resolution: Not a Problem So I think this is basically NotAProblem in the sense that it's ToBeExpected

[jira] [Commented] (SPARK-6050) Spark on YARN does not work --executor-cores is specified

[ https://issues.apache.org/jira/browse/SPARK-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340195#comment-14340195 ] Thomas Graves commented on SPARK-6050: -- Thanks for investigating this more. This is

[jira] [Commented] (SPARK-4900) MLlib SingularValueDecomposition ARPACK IllegalStateException

[ https://issues.apache.org/jira/browse/SPARK-4900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340202#comment-14340202 ] Sean Owen commented on SPARK-4900: -- Hm, I tend to agree that upfront error checking is

[jira] [Commented] (SPARK-5081) Shuffle write increases

[ https://issues.apache.org/jira/browse/SPARK-5081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340089#comment-14340089 ] Dr. Christian Betz commented on SPARK-5081: --- Ok, I can really bring the Thread

[jira] [Resolved] (SPARK-6033) the description abou the spark.worker.cleanup.enabled is not matched with the code

[ https://issues.apache.org/jira/browse/SPARK-6033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6033. -- Resolution: Fixed Fix Version/s: 1.2.2 1.1.2 1.3.0

[jira] [Resolved] (SPARK-5917) Distinct is broken

[ https://issues.apache.org/jira/browse/SPARK-5917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5917. -- Resolution: Duplicate Resolving with the most likely explanation; reopen if there is evidence to the

[jira] [Comment Edited] (SPARK-6049) HiveThriftServer2 may expose Inheritable methods

[ https://issues.apache.org/jira/browse/SPARK-6049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340096#comment-14340096 ] Littlestar edited comment on SPARK-6049 at 2/27/15 2:43 PM:

[jira] [Updated] (SPARK-5081) Shuffle write increases

[ https://issues.apache.org/jira/browse/SPARK-5081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr. Christian Betz updated SPARK-5081: -- Attachment: diff.txt And diff.txt is my diff from CDH-version to pure-version. Sorry,

[jira] [Updated] (SPARK-6058) Log the error for the EXIT_EXCEPTION_USER_CLASS exit code

[ https://issues.apache.org/jira/browse/SPARK-6058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6058: - Assignee: Shixiong Zhu Log the error for the EXIT_EXCEPTION_USER_CLASS exit code

[jira] [Resolved] (SPARK-6059) Add volatile to ApplicationMaster.reporterThread and ApplicationMaster.allocator

[ https://issues.apache.org/jira/browse/SPARK-6059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6059. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 4814

[jira] [Resolved] (SPARK-6058) Log the error for the EXIT_EXCEPTION_USER_CLASS exit code

[ https://issues.apache.org/jira/browse/SPARK-6058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6058. -- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4813

[jira] [Commented] (SPARK-4900) MLlib SingularValueDecomposition ARPACK IllegalStateException

[ https://issues.apache.org/jira/browse/SPARK-4900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340208#comment-14340208 ] Mike Beyer commented on SPARK-4900: --- in my opinion there should be no automatic error

[jira] [Resolved] (SPARK-5739) Size exceeds Integer.MAX_VALUE in File Map

[ https://issues.apache.org/jira/browse/SPARK-5739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5739. -- Resolution: Duplicate I think at best this reduces to just hitting the issue that blocks can't be 2GB

[jira] [Updated] (SPARK-5434) Preserve spaces in path to spark-ec2

[ https://issues.apache.org/jira/browse/SPARK-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5434: - Labels: (was: backport-needed) Preserve spaces in path to spark-ec2

[jira] [Updated] (SPARK-6054) SQL UDF returning object of case class; regression from 1.2.0

[ https://issues.apache.org/jira/browse/SPARK-6054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6054: - Component/s: SQL SQL UDF returning object of case class; regression from 1.2.0

[jira] [Resolved] (SPARK-5615) Fix testPackage in StreamingContextSuite

[ https://issues.apache.org/jira/browse/SPARK-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5615. -- Resolution: Duplicate OK, this was superseded by / subsumed by the follow up issue SPARK-5681 Fix

[jira] [Updated] (SPARK-5570) No docs stating that `new SparkConf().set(spark.driver.memory, ...) will not work

[ https://issues.apache.org/jira/browse/SPARK-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5570: - Component/s: (was: Spark Core) Priority: Minor (was: Major) Target Version/s:

[jira] [Resolved] (SPARK-5570) No docs stating that `new SparkConf().set(spark.driver.memory, ...) will not work

[ https://issues.apache.org/jira/browse/SPARK-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5570. -- Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Ilya Ganelin (was: Andrew Or) No

[jira] [Updated] (SPARK-6059) Add volatile to ApplicationMaster.reporterThread and ApplicationMaster.allocator

[ https://issues.apache.org/jira/browse/SPARK-6059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6059: - Assignee: Shixiong Zhu Add volatile to ApplicationMaster.reporterThread and

[jira] [Updated] (SPARK-6063) MLlib doesn't pass mvn scalastyle check due to UTF chars in LDAModel.scala

[ https://issues.apache.org/jira/browse/SPARK-6063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Griffiths updated SPARK-6063: - Description: On Windows 8.1, trying to build Spark from source (latest Github pull)

[jira] [Commented] (SPARK-6063) MLlib doesn't pass mvn scalastyle check due to UTF chars in LDAModel.scala

[ https://issues.apache.org/jira/browse/SPARK-6063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340513#comment-14340513 ] Apache Spark commented on SPARK-6063: - User 'msjgriffiths' has created a pull request

[jira] [Updated] (SPARK-6063) MLlib doesn't pass mvn scalastyle check due to UTF chars in LDAModel.scala

[ https://issues.apache.org/jira/browse/SPARK-6063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Griffiths updated SPARK-6063: - Description: On Windows 8.1, trying to build Spark from source (latest Github pull)

[jira] [Commented] (SPARK-4587) Model export/import

[ https://issues.apache.org/jira/browse/SPARK-4587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340539#comment-14340539 ] Apache Spark commented on SPARK-4587: - User 'jkbradley' has created a pull request for

[jira] [Created] (SPARK-6063) MLlib doesn't pass mvn scalastyle check due to UTF chars in LDAModel.scala

Michael Griffiths created SPARK-6063: Summary: MLlib doesn't pass mvn scalastyle check due to UTF chars in LDAModel.scala Key: SPARK-6063 URL: https://issues.apache.org/jira/browse/SPARK-6063

[jira] [Updated] (SPARK-6064) Checking data types when resolving types

[ https://issues.apache.org/jira/browse/SPARK-6064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zeng updated SPARK-6064: Shepherd: Yin Huai Checking data types when resolving types

[jira] [Updated] (SPARK-6055) Memory leak in pyspark sql due to incorrect equality check

[ https://issues.apache.org/jira/browse/SPARK-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-6055: --- Summary: Memory leak in pyspark sql due to incorrect equality check (was: memory leak in

[jira] [Updated] (SPARK-6063) MLlib doesn't pass mvn scalastyle check due to UTF chars in LDAModel.scala

[ https://issues.apache.org/jira/browse/SPARK-6063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Griffiths updated SPARK-6063: - Description: On Windows 8.1, trying to build Spark from source (latest Github pull)

[jira] [Updated] (SPARK-6063) MLlib doesn't pass mvn scalastyle check due to UTF chars in LDAModel.scala

[ https://issues.apache.org/jira/browse/SPARK-6063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Griffiths updated SPARK-6063: - Description: On Windows 8.1, trying to build Spark from source (latest Github pull)

[jira] [Created] (SPARK-6064) Checking data types when resolving types

Kai Zeng created SPARK-6064: --- Summary: Checking data types when resolving types Key: SPARK-6064 URL: https://issues.apache.org/jira/browse/SPARK-6064 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-5297) JavaStreamingContext.fileStream won't work because type info isn't propagated

[ https://issues.apache.org/jira/browse/SPARK-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5297: - Labels: (was: backport-needed) JavaStreamingContext.fileStream won't work because type info isn't

[jira] [Commented] (SPARK-4900) MLlib SingularValueDecomposition ARPACK IllegalStateException

[ https://issues.apache.org/jira/browse/SPARK-4900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340278#comment-14340278 ] Sean Owen commented on SPARK-4900: -- [~mengxr] what do you think on this one? I/we could

[jira] [Commented] (SPARK-5654) Integrate SparkR into Apache Spark

[ https://issues.apache.org/jira/browse/SPARK-5654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340294#comment-14340294 ] Hari Sekhon commented on SPARK-5654: Sean - ever worked for a bank? What you've said

[jira] [Commented] (SPARK-5654) Integrate SparkR into Apache Spark

[ https://issues.apache.org/jira/browse/SPARK-5654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340299#comment-14340299 ] Sean Owen commented on SPARK-5654: -- [~harisekhon] Yes, we work for banks as you know. I

[jira] [Commented] (SPARK-4900) MLlib SingularValueDecomposition ARPACK IllegalStateException

[ https://issues.apache.org/jira/browse/SPARK-4900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340229#comment-14340229 ] Sean Owen commented on SPARK-4900: -- This is still mostly counting on the user to validate

[jira] [Updated] (SPARK-5417) Remove redundant executor-ID set() call

[ https://issues.apache.org/jira/browse/SPARK-5417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5417: - Labels: (was: backport-needed) Remove redundant executor-ID set() call

[jira] [Commented] (SPARK-4900) MLlib SingularValueDecomposition ARPACK IllegalStateException

[ https://issues.apache.org/jira/browse/SPARK-4900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340235#comment-14340235 ] Mike Beyer commented on SPARK-4900: --- I would suggest a

[jira] [Commented] (SPARK-5417) Remove redundant executor-ID set() call

[ https://issues.apache.org/jira/browse/SPARK-5417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340268#comment-14340268 ] Sean Owen commented on SPARK-5417: -- Backported to 1.2 as well. Remove redundant

[jira] [Commented] (SPARK-4814) Enable assertions in SBT, Maven tests / AssertionError from Hive's LazyBinaryInteger

[ https://issues.apache.org/jira/browse/SPARK-4814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340275#comment-14340275 ] Sean Owen commented on SPARK-4814: -- General question on back-ports: at this stage, is it

[jira] [Resolved] (SPARK-5297) JavaStreamingContext.fileStream won't work because type info isn't propagated

[ https://issues.apache.org/jira/browse/SPARK-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5297. -- Resolution: Fixed Target Version/s: (was: 1.3.0, 1.2.1) JavaStreamingContext.fileStream

[jira] [Resolved] (SPARK-5417) Remove redundant executor-ID set() call

[ https://issues.apache.org/jira/browse/SPARK-5417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5417. -- Resolution: Fixed Fix Version/s: 1.2.2 Target Version/s: (was: 1.3.0, 1.2.1) Remove

[jira] [Commented] (SPARK-5297) JavaStreamingContext.fileStream won't work because type info isn't propagated

[ https://issues.apache.org/jira/browse/SPARK-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340271#comment-14340271 ] Sean Owen commented on SPARK-5297: -- Decided not to back port to 1.2 per tdas

[jira] [Commented] (SPARK-6050) Spark on YARN does not work --executor-cores is specified

[ https://issues.apache.org/jira/browse/SPARK-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340773#comment-14340773 ] Thomas Graves commented on SPARK-6050: -- Sounds like you figured it out already, I cut

[jira] [Commented] (SPARK-6050) Spark on YARN does not work --executor-cores is specified

[ https://issues.apache.org/jira/browse/SPARK-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340707#comment-14340707 ] Marcelo Vanzin commented on SPARK-6050: --- Oh, let me try again with

[jira] [Created] (SPARK-6066) Metadata in event log makes it very difficult for external libraries to parse event log

Kay Ousterhout created SPARK-6066: - Summary: Metadata in event log makes it very difficult for external libraries to parse event log Key: SPARK-6066 URL: https://issues.apache.org/jira/browse/SPARK-6066

[jira] [Commented] (SPARK-6048) SparkConf.translateConfKey should translate on get, not set

[ https://issues.apache.org/jira/browse/SPARK-6048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340740#comment-14340740 ] Patrick Wendell commented on SPARK-6048: Okay I just talked to [~vanzin] offline.

[jira] [Created] (SPARK-6065) Optimize word2vec.findSynonyms speed

Joseph K. Bradley created SPARK-6065: Summary: Optimize word2vec.findSynonyms speed Key: SPARK-6065 URL: https://issues.apache.org/jira/browse/SPARK-6065 Project: Spark Issue Type:

  1   2   >