[jira] [Updated] (SPARK-17110) Pyspark with locality ANY throw java.io.StreamCorruptedException

[ https://issues.apache.org/jira/browse/SPARK-17110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomer Kaftan updated SPARK-17110: - Environment: Cluster of 2 AWS r3.xlarge slaves launched via ec2 scripts, Spark 2.0.0, hadoop: yar

[jira] [Commented] (SPARK-17110) Pyspark with locality ANY throw java.io.StreamCorruptedException

[ https://issues.apache.org/jira/browse/SPARK-17110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440817#comment-15440817 ] Tomer Kaftan commented on SPARK-17110: -- Hi Miao, That setup wouldn't cause this bu

[jira] [Commented] (SPARK-17276) Stop environment parameters flooding Jenkins build output

[ https://issues.apache.org/jira/browse/SPARK-17276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440812#comment-15440812 ] Apache Spark commented on SPARK-17276: -- User 'keypointt' has created a pull request

[jira] [Assigned] (SPARK-17276) Stop environment parameters flooding Jenkins build output

[ https://issues.apache.org/jira/browse/SPARK-17276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17276: Assignee: (was: Apache Spark) > Stop environment parameters flooding Jenkins build out

[jira] [Assigned] (SPARK-17276) Stop environment parameters flooding Jenkins build output

[ https://issues.apache.org/jira/browse/SPARK-17276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17276: Assignee: Apache Spark > Stop environment parameters flooding Jenkins build output > -

[jira] [Assigned] (SPARK-17254) Filter operator should have “stop if false” semantics for sorted data

[ https://issues.apache.org/jira/browse/SPARK-17254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17254: Assignee: Apache Spark > Filter operator should have “stop if false” semantics for sorted

[jira] [Commented] (SPARK-17254) Filter operator should have “stop if false” semantics for sorted data

[ https://issues.apache.org/jira/browse/SPARK-17254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440806#comment-15440806 ] Apache Spark commented on SPARK-17254: -- User 'viirya' has created a pull request for

[jira] [Assigned] (SPARK-17254) Filter operator should have “stop if false” semantics for sorted data

[ https://issues.apache.org/jira/browse/SPARK-17254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17254: Assignee: (was: Apache Spark) > Filter operator should have “stop if false” semantics

[jira] [Updated] (SPARK-17110) Pyspark with locality ANY throw java.io.StreamCorruptedException

[ https://issues.apache.org/jira/browse/SPARK-17110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomer Kaftan updated SPARK-17110: - Description: In Pyspark 2.0.0, any task that accesses cached data non-locally throws a StreamCor

[jira] [Commented] (SPARK-17110) Pyspark with locality ANY throw java.io.StreamCorruptedException

[ https://issues.apache.org/jira/browse/SPARK-17110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440786#comment-15440786 ] Miao Wang commented on SPARK-17110: --- I set up a two-node cluster, one master, one worke

[jira] [Updated] (SPARK-17276) Stop environment parameters flooding Jenkins build output

[ https://issues.apache.org/jira/browse/SPARK-17276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Ren updated SPARK-17276: Attachment: Screen Shot 2016-08-26 at 10.52.07 PM.png > Stop environment parameters flooding Jenkins build

[jira] [Commented] (SPARK-17276) Stop environment parameters flooding Jenkins build output

[ https://issues.apache.org/jira/browse/SPARK-17276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440782#comment-15440782 ] Xin Ren commented on SPARK-17276: - I'm working on it. > Stop environment parameters floo

[jira] [Created] (SPARK-17276) Stop environment parameters flooding Jenkins build output

Xin Ren created SPARK-17276: --- Summary: Stop environment parameters flooding Jenkins build output Key: SPARK-17276 URL: https://issues.apache.org/jira/browse/SPARK-17276 Project: Spark Issue Type: I

[jira] [Commented] (SPARK-17274) Move join optimizer rules into a separate file

[ https://issues.apache.org/jira/browse/SPARK-17274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440750#comment-15440750 ] Apache Spark commented on SPARK-17274: -- User 'rxin' has created a pull request for t

[jira] [Assigned] (SPARK-17274) Move join optimizer rules into a separate file

[ https://issues.apache.org/jira/browse/SPARK-17274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17274: Assignee: Reynold Xin (was: Apache Spark) > Move join optimizer rules into a separate fil

[jira] [Assigned] (SPARK-17274) Move join optimizer rules into a separate file

[ https://issues.apache.org/jira/browse/SPARK-17274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17274: Assignee: Apache Spark (was: Reynold Xin) > Move join optimizer rules into a separate fil

[jira] [Commented] (SPARK-17275) Flaky test: org.apache.spark.deploy.RPackageUtilsSuite.jars that don't exist are skipped and print warning

[ https://issues.apache.org/jira/browse/SPARK-17275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440747#comment-15440747 ] Yin Huai commented on SPARK-17275: -- cc [~felixcheung] [~shivaram] > Flaky test: org.apa

[jira] [Created] (SPARK-17275) Flaky test: org.apache.spark.deploy.RPackageUtilsSuite.jars that don't exist are skipped and print warning

Yin Huai created SPARK-17275: Summary: Flaky test: org.apache.spark.deploy.RPackageUtilsSuite.jars that don't exist are skipped and print warning Key: SPARK-17275 URL: https://issues.apache.org/jira/browse/SPARK-1727

[jira] [Created] (SPARK-17274) Move join optimizer rules into a separate file

Reynold Xin created SPARK-17274: --- Summary: Move join optimizer rules into a separate file Key: SPARK-17274 URL: https://issues.apache.org/jira/browse/SPARK-17274 Project: Spark Issue Type: Sub-

[jira] [Assigned] (SPARK-17273) Move expression optimizer rules into a separate file

[ https://issues.apache.org/jira/browse/SPARK-17273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17273: Assignee: Reynold Xin (was: Apache Spark) > Move expression optimizer rules into a separa

[jira] [Commented] (SPARK-17273) Move expression optimizer rules into a separate file

[ https://issues.apache.org/jira/browse/SPARK-17273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440742#comment-15440742 ] Apache Spark commented on SPARK-17273: -- User 'rxin' has created a pull request for t

[jira] [Assigned] (SPARK-17273) Move expression optimizer rules into a separate file

[ https://issues.apache.org/jira/browse/SPARK-17273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17273: Assignee: Apache Spark (was: Reynold Xin) > Move expression optimizer rules into a separa

[jira] [Assigned] (SPARK-17272) Move subquery optimizer rules into its own file

[ https://issues.apache.org/jira/browse/SPARK-17272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17272: Assignee: Apache Spark (was: Reynold Xin) > Move subquery optimizer rules into its own fi

[jira] [Assigned] (SPARK-17272) Move subquery optimizer rules into its own file

[ https://issues.apache.org/jira/browse/SPARK-17272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17272: Assignee: Reynold Xin (was: Apache Spark) > Move subquery optimizer rules into its own fi

[jira] [Created] (SPARK-17273) Move expression optimizer rules into a separate file

Reynold Xin created SPARK-17273: --- Summary: Move expression optimizer rules into a separate file Key: SPARK-17273 URL: https://issues.apache.org/jira/browse/SPARK-17273 Project: Spark Issue Type

[jira] [Commented] (SPARK-17272) Move subquery optimizer rules into its own file

[ https://issues.apache.org/jira/browse/SPARK-17272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440733#comment-15440733 ] Apache Spark commented on SPARK-17272: -- User 'rxin' has created a pull request for t

[jira] [Created] (SPARK-17272) Move subquery optimizer rules into its own file

Reynold Xin created SPARK-17272: --- Summary: Move subquery optimizer rules into its own file Key: SPARK-17272 URL: https://issues.apache.org/jira/browse/SPARK-17272 Project: Spark Issue Type: Sub

[jira] [Commented] (SPARK-17270) Move object optimization rules into its own file

[ https://issues.apache.org/jira/browse/SPARK-17270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440725#comment-15440725 ] Apache Spark commented on SPARK-17270: -- User 'rxin' has created a pull request for t

[jira] [Resolved] (SPARK-17269) Move finish analysis stage into its own file

[ https://issues.apache.org/jira/browse/SPARK-17269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17269. - Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 > Move finish analysis sta

[jira] [Resolved] (SPARK-17270) Move object optimization rules into its own file

[ https://issues.apache.org/jira/browse/SPARK-17270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-17270. -- Resolution: Fixed Fix Version/s: 2.1.0 https://github.com/apache/spark/pull/14839 has been merge

[jira] [Updated] (SPARK-10747) add support for NULLS FIRST|LAST in ORDER BY clause

[ https://issues.apache.org/jira/browse/SPARK-10747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Wu updated SPARK-10747: --- Summary: add support for NULLS FIRST|LAST in ORDER BY clause (was: add support for window specification to i

[jira] [Updated] (SPARK-17268) Break Optimizer.scala apart

[ https://issues.apache.org/jira/browse/SPARK-17268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17268: Description: Optimizer.scala has become too large to maintain. We would need to break it apart int

[jira] [Commented] (SPARK-10747) add support for window specification to include how NULLS are ordered

[ https://issues.apache.org/jira/browse/SPARK-10747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440656#comment-15440656 ] Xin Wu commented on SPARK-10747: This JIRA may be changed to support NULLS FIRST|LAST fea

[jira] [Assigned] (SPARK-10747) add support for window specification to include how NULLS are ordered

[ https://issues.apache.org/jira/browse/SPARK-10747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10747: Assignee: Apache Spark > add support for window specification to include how NULLS are ord

[jira] [Commented] (SPARK-10747) add support for window specification to include how NULLS are ordered

[ https://issues.apache.org/jira/browse/SPARK-10747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440651#comment-15440651 ] Apache Spark commented on SPARK-10747: -- User 'xwu0226' has created a pull request fo

[jira] [Assigned] (SPARK-10747) add support for window specification to include how NULLS are ordered

[ https://issues.apache.org/jira/browse/SPARK-10747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10747: Assignee: (was: Apache Spark) > add support for window specification to include how NU

[jira] [Updated] (SPARK-10747) add support for window specification to include how NULLS are ordered

[ https://issues.apache.org/jira/browse/SPARK-10747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Wu updated SPARK-10747: --- Issue Type: New Feature (was: Improvement) > add support for window specification to include how NULLS are o

[jira] [Commented] (SPARK-16963) Change Source API so that sources do not need to keep unbounded state

[ https://issues.apache.org/jira/browse/SPARK-16963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440598#comment-15440598 ] Frederick Reiss commented on SPARK-16963: - Updated the pull request to address so

[jira] [Assigned] (SPARK-17271) Planner adds un-necessary Sort even if child ordering is semantically same as required ordering

[ https://issues.apache.org/jira/browse/SPARK-17271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17271: Assignee: Apache Spark > Planner adds un-necessary Sort even if child ordering is semantic

[jira] [Assigned] (SPARK-17271) Planner adds un-necessary Sort even if child ordering is semantically same as required ordering

[ https://issues.apache.org/jira/browse/SPARK-17271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17271: Assignee: (was: Apache Spark) > Planner adds un-necessary Sort even if child ordering

[jira] [Commented] (SPARK-17271) Planner adds un-necessary Sort even if child ordering is semantically same as required ordering

[ https://issues.apache.org/jira/browse/SPARK-17271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440557#comment-15440557 ] Apache Spark commented on SPARK-17271: -- User 'tejasapatil' has created a pull reques

[jira] [Updated] (SPARK-17271) Planner adds un-necessary Sort even if child ordering is semantically same as required ordering

[ https://issues.apache.org/jira/browse/SPARK-17271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tejas Patil updated SPARK-17271: Description: Found a case when the planner is adding un-needed SORT operation due to bug in the wa

[jira] [Commented] (SPARK-16216) CSV data source does not write date and timestamp correctly

[ https://issues.apache.org/jira/browse/SPARK-16216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440546#comment-15440546 ] Apache Spark commented on SPARK-16216: -- User 'HyukjinKwon' has created a pull reques

[jira] [Resolved] (SPARK-17266) PrefixComparatorsSuite's "String prefix comparator" failed when both input strings are empty strings

[ https://issues.apache.org/jira/browse/SPARK-17266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-17266. -- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14837 [https://github.com/

[jira] [Commented] (SPARK-13525) SparkR: java.net.SocketTimeoutException: Accept timed out when running any dataframe function

[ https://issues.apache.org/jira/browse/SPARK-13525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440505#comment-15440505 ] Sun Rui commented on SPARK-13525: - What's your spark cluster deployment mode? yarn or sta

[jira] [Updated] (SPARK-17266) PrefixComparatorsSuite's "String prefix comparator" failed when both input strings are empty strings

[ https://issues.apache.org/jira/browse/SPARK-17266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-17266: - Assignee: Yin Huai > PrefixComparatorsSuite's "String prefix comparator" failed when both input > string

[jira] [Created] (SPARK-17271) Planner adds un-necessary Sort even if child ordering is semantically same as required ordering

Tejas Patil created SPARK-17271: --- Summary: Planner adds un-necessary Sort even if child ordering is semantically same as required ordering Key: SPARK-17271 URL: https://issues.apache.org/jira/browse/SPARK-17271

[jira] [Commented] (SPARK-13525) SparkR: java.net.SocketTimeoutException: Accept timed out when running any dataframe function

[ https://issues.apache.org/jira/browse/SPARK-13525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440476#comment-15440476 ] Sun Rui commented on SPARK-13525: - Another guess: could you check "localhost" works for

[jira] [Commented] (SPARK-17270) Move object optimization rules into its own file

[ https://issues.apache.org/jira/browse/SPARK-17270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440368#comment-15440368 ] Apache Spark commented on SPARK-17270: -- User 'rxin' has created a pull request for t

[jira] [Assigned] (SPARK-17270) Move object optimization rules into its own file

[ https://issues.apache.org/jira/browse/SPARK-17270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17270: Assignee: Reynold Xin (was: Apache Spark) > Move object optimization rules into its own f

[jira] [Assigned] (SPARK-17270) Move object optimization rules into its own file

[ https://issues.apache.org/jira/browse/SPARK-17270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17270: Assignee: Apache Spark (was: Reynold Xin) > Move object optimization rules into its own f

[jira] [Created] (SPARK-17270) Move object optimization rules into its own file

Reynold Xin created SPARK-17270: --- Summary: Move object optimization rules into its own file Key: SPARK-17270 URL: https://issues.apache.org/jira/browse/SPARK-17270 Project: Spark Issue Type: Su

[jira] [Commented] (SPARK-17269) Move finish analysis stage into its own file

[ https://issues.apache.org/jira/browse/SPARK-17269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440359#comment-15440359 ] Apache Spark commented on SPARK-17269: -- User 'rxin' has created a pull request for t

[jira] [Assigned] (SPARK-17269) Move finish analysis stage into its own file

[ https://issues.apache.org/jira/browse/SPARK-17269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17269: Assignee: Reynold Xin (was: Apache Spark) > Move finish analysis stage into its own file

[jira] [Assigned] (SPARK-17269) Move finish analysis stage into its own file

[ https://issues.apache.org/jira/browse/SPARK-17269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17269: Assignee: Apache Spark (was: Reynold Xin) > Move finish analysis stage into its own file

[jira] [Created] (SPARK-17269) Move finish analysis stage into its own file

Reynold Xin created SPARK-17269: --- Summary: Move finish analysis stage into its own file Key: SPARK-17269 URL: https://issues.apache.org/jira/browse/SPARK-17269 Project: Spark Issue Type: Sub-ta

[jira] [Created] (SPARK-17268) Break Optimizer.scala apart

Reynold Xin created SPARK-17268: --- Summary: Break Optimizer.scala apart Key: SPARK-17268 URL: https://issues.apache.org/jira/browse/SPARK-17268 Project: Spark Issue Type: Improvement C

[jira] [Resolved] (SPARK-17244) Joins should not pushdown non-deterministic conditions

[ https://issues.apache.org/jira/browse/SPARK-17244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-17244. -- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull request 1

[jira] [Updated] (SPARK-17244) Joins should not pushdown non-deterministic conditions

[ https://issues.apache.org/jira/browse/SPARK-17244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-17244: - Assignee: Sameer Agarwal > Joins should not pushdown non-deterministic conditions > -

[jira] [Commented] (SPARK-17163) Merge MLOR into a single LOR interface

[ https://issues.apache.org/jira/browse/SPARK-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440183#comment-15440183 ] DB Tsai commented on SPARK-17163: - It relates to this [SPARK-17201], but seems that it's

[jira] [Commented] (SPARK-17266) PrefixComparatorsSuite's "String prefix comparator" failed when both input strings are empty strings

[ https://issues.apache.org/jira/browse/SPARK-17266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440146#comment-15440146 ] Apache Spark commented on SPARK-17266: -- User 'yhuai' has created a pull request for

[jira] [Assigned] (SPARK-17266) PrefixComparatorsSuite's "String prefix comparator" failed when both input strings are empty strings

[ https://issues.apache.org/jira/browse/SPARK-17266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17266: Assignee: (was: Apache Spark) > PrefixComparatorsSuite's "String prefix comparator" fa

[jira] [Assigned] (SPARK-17266) PrefixComparatorsSuite's "String prefix comparator" failed when both input strings are empty strings

[ https://issues.apache.org/jira/browse/SPARK-17266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17266: Assignee: Apache Spark > PrefixComparatorsSuite's "String prefix comparator" failed when b

[jira] [Commented] (SPARK-17163) Merge MLOR into a single LOR interface

[ https://issues.apache.org/jira/browse/SPARK-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440142#comment-15440142 ] Joseph K. Bradley commented on SPARK-17163: --- I was guessing that optimization w

[jira] [Updated] (SPARK-17267) Long running structured streaming requirements

[ https://issues.apache.org/jira/browse/SPARK-17267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17267: Priority: Blocker (was: Major) > Long running structured streaming requirements >

[jira] [Updated] (SPARK-15698) Ability to remove old metadata for structure streaming MetadataLog

[ https://issues.apache.org/jira/browse/SPARK-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-15698: Issue Type: Sub-task (was: Improvement) Parent: SPARK-17267 > Ability to remove old metada

[jira] [Updated] (SPARK-17235) MetadataLog should support purging old logs

[ https://issues.apache.org/jira/browse/SPARK-17235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17235: Issue Type: Sub-task (was: New Feature) Parent: SPARK-17267 > MetadataLog should support p

[jira] [Updated] (SPARK-17165) FileStreamSource should not track the list of seen files indefinitely

[ https://issues.apache.org/jira/browse/SPARK-17165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17165: Issue Type: Sub-task (was: Bug) Parent: SPARK-17267 > FileStreamSource should not track th

[jira] [Created] (SPARK-17267) Long running structured streaming requirements

Reynold Xin created SPARK-17267: --- Summary: Long running structured streaming requirements Key: SPARK-17267 URL: https://issues.apache.org/jira/browse/SPARK-17267 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-17235) MetadataLog should support purging old logs

[ https://issues.apache.org/jira/browse/SPARK-17235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17235. - Resolution: Fixed Assignee: Peter Lee Fix Version/s: 2.1.0 2.0.

[jira] [Created] (SPARK-17266) PrefixComparatorsSuite's "String prefix comparator" failed when both input strings are empty strings

Yin Huai created SPARK-17266: Summary: PrefixComparatorsSuite's "String prefix comparator" failed when both input strings are empty strings Key: SPARK-17266 URL: https://issues.apache.org/jira/browse/SPARK-17266

[jira] [Commented] (SPARK-13525) SparkR: java.net.SocketTimeoutException: Accept timed out when running any dataframe function

[ https://issues.apache.org/jira/browse/SPARK-13525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440090#comment-15440090 ] Arihanth Jain commented on SPARK-13525: --- [~sunrui] I have tried "spark.sparkr.use.d

[jira] [Commented] (SPARK-17044) Add window function test in SQLQueryTestSuite

[ https://issues.apache.org/jira/browse/SPARK-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15439925#comment-15439925 ] Dongjoon Hyun commented on SPARK-17044: --- Hi, [~rxin]. Could you review this issue?

[jira] [Updated] (SPARK-17044) Add window function test in SQLQueryTestSuite

[ https://issues.apache.org/jira/browse/SPARK-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-17044: -- Description: This issue adds a SQL query test for Window functions for new `SQLQueryTestSuite`.

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15439857#comment-15439857 ] Apache Spark commented on SPARK-17243: -- User 'ajbozarth' has created a pull request

[jira] [Assigned] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17243: Assignee: (was: Apache Spark) > Spark 2.0 history server summary page gets stuck at "l

[jira] [Assigned] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17243: Assignee: Apache Spark > Spark 2.0 history server summary page gets stuck at "loading hist

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15439799#comment-15439799 ] Alex Bozarth commented on SPARK-17243: -- So I decided to work on this as a short brea

[jira] [Commented] (SPARK-15882) Discuss distributed linear algebra in spark.ml package

[ https://issues.apache.org/jira/browse/SPARK-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15439767#comment-15439767 ] Joseph K. Bradley commented on SPARK-15882: --- This seems like an important featu

[jira] [Resolved] (SPARK-17246) Support BigDecimal literal parsing

[ https://issues.apache.org/jira/browse/SPARK-17246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17246. - Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 > Support BigDecimal liter

[jira] [Resolved] (SPARK-16967) Collect Mesos support code into a module/profile

[ https://issues.apache.org/jira/browse/SPARK-16967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-16967. Resolution: Fixed Assignee: Michael Gummelt Fix Version/s: 2.1.0 > Collect

[jira] [Created] (SPARK-17265) EdgeRDD Difference throws an exception

Shishir Kharel created SPARK-17265: -- Summary: EdgeRDD Difference throws an exception Key: SPARK-17265 URL: https://issues.apache.org/jira/browse/SPARK-17265 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-17261) Using HiveContext after re-creating SparkContext in Spark 2.0 throws "Java.lang.illegalStateException: Cannot call methods on a stopped sparkContext"

[ https://issues.apache.org/jira/browse/SPARK-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15439613#comment-15439613 ] Dongjoon Hyun commented on SPARK-17261: --- Hi, [~dakghar] For me, those seems not to

[jira] [Updated] (SPARK-17220) Upgrade Py4J to 0.10.3

[ https://issues.apache.org/jira/browse/SPARK-17220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-17220: - Component/s: PySpark > Upgrade Py4J to 0.10.3 > -- > >

[jira] [Updated] (SPARK-17265) EdgeRDD Difference throws an exception

[ https://issues.apache.org/jira/browse/SPARK-17265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shishir Kharel updated SPARK-17265: --- Description: Subtracting two edge RDD throws and exception. val difference = graph1.edges.su

[jira] [Commented] (SPARK-16501) spark.mesos.secret exposed on UI and command line

[ https://issues.apache.org/jira/browse/SPARK-16501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15439609#comment-15439609 ] Eric Daniel commented on SPARK-16501: - Great to know, thanks! > spark.mesos.secret e

[jira] [Resolved] (SPARK-17207) Comparing Vector in relative tolerance or absolute tolerance in UnitTests error

[ https://issues.apache.org/jira/browse/SPARK-17207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-17207. - Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14785 [https://github.com/ap

[jira] [Updated] (SPARK-17165) FileStreamSource should not track the list of seen files indefinitely

[ https://issues.apache.org/jira/browse/SPARK-17165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17165: - Fix Version/s: 2.1.0 2.0.1 > FileStreamSource should not track the list of see

[jira] [Resolved] (SPARK-17165) FileStreamSource should not track the list of seen files indefinitely

[ https://issues.apache.org/jira/browse/SPARK-17165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17165. -- Resolution: Fixed Assignee: Peter Lee > FileStreamSource should not track the list of see

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15439517#comment-15439517 ] Gang Wu commented on SPARK-17243: - Thanks [~ajbozarth]! Let me know when it is done. > S

[jira] [Updated] (SPARK-17250) Remove HiveClient and setCurrentDatabase from HiveSessionCatalog

[ https://issues.apache.org/jira/browse/SPARK-17250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-17250: - Assignee: Xiao Li (was: Apache Spark) > Remove HiveClient and setCurrentDatabase from HiveSessionCatalog

[jira] [Resolved] (SPARK-17250) Remove HiveClient and setCurrentDatabase from HiveSessionCatalog

[ https://issues.apache.org/jira/browse/SPARK-17250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-17250. -- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14821 [https://github.com/

[jira] [Commented] (SPARK-17163) Merge MLOR into a single LOR interface

[ https://issues.apache.org/jira/browse/SPARK-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15439499#comment-15439499 ] Seth Hendrickson commented on SPARK-17163: -- [~dbtsai] We can discuss these desig

[jira] [Updated] (SPARK-17192) Issuing an exception when users specify the partitioning columns without a given schema

[ https://issues.apache.org/jira/browse/SPARK-17192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-17192: - Assignee: Xiao Li > Issuing an exception when users specify the partitioning columns without a > given s

[jira] [Resolved] (SPARK-17192) Issuing an exception when users specify the partitioning columns without a given schema

[ https://issues.apache.org/jira/browse/SPARK-17192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-17192. -- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14572 [https://github.com/

[jira] [Commented] (SPARK-17252) Performing arithmetic in VALUES can lead to ClassCastException / MatchErrors during query parsing

[ https://issues.apache.org/jira/browse/SPARK-17252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15439478#comment-15439478 ] Josh Rosen commented on SPARK-17252: Looks like this issue only affects 2.0.0, so I'm

[jira] [Resolved] (SPARK-17252) Performing arithmetic in VALUES can lead to ClassCastException / MatchErrors during query parsing

[ https://issues.apache.org/jira/browse/SPARK-17252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-17252. Resolution: Fixed Fix Version/s: 2.0.1 > Performing arithmetic in VALUES can lead to ClassCa

[jira] [Commented] (SPARK-17163) Merge MLOR into a single LOR interface

[ https://issues.apache.org/jira/browse/SPARK-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15439475#comment-15439475 ] Apache Spark commented on SPARK-17163: -- User 'sethah' has created a pull request for

[jira] [Assigned] (SPARK-17163) Merge MLOR into a single LOR interface

[ https://issues.apache.org/jira/browse/SPARK-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17163: Assignee: Apache Spark > Merge MLOR into a single LOR interface >

[jira] [Assigned] (SPARK-17163) Merge MLOR into a single LOR interface

[ https://issues.apache.org/jira/browse/SPARK-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17163: Assignee: (was: Apache Spark) > Merge MLOR into a single LOR interface > -

  1   2   >