[jira] [Comment Edited] (SPARK-8824) Support Parquet logical types TIMESTAMP_MILLIS and TIMESTAMP_MICROS

2015-08-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681331#comment-14681331 ] Cheng Lian edited comment on SPARK-8824 at 8/11/15 6:55 AM: Oh

[jira] [Commented] (SPARK-8824) Support Parquet logical types TIMESTAMP_MILLIS and TIMESTAMP_MICROS

2015-08-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681331#comment-14681331 ] Cheng Lian commented on SPARK-8824: --- Oh sorry, I mistook your request for {{TIMESTAMP_MI

[jira] [Comment Edited] (SPARK-9813) Incorrect UNION ALL behavior

2015-08-10 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681326#comment-14681326 ] Simeon Simeonov edited comment on SPARK-9813 at 8/11/15 6:47 AM: ---

[jira] [Commented] (SPARK-9813) Incorrect UNION ALL behavior

2015-08-10 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681326#comment-14681326 ] Simeon Simeonov commented on SPARK-9813: [~hvanhovell] Oracle requires the number

[jira] [Comment Edited] (SPARK-9813) Incorrect UNION ALL behavior

2015-08-10 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681326#comment-14681326 ] Simeon Simeonov edited comment on SPARK-9813 at 8/11/15 6:46 AM: ---

[jira] [Resolved] (SPARK-9640) Do not run Python Kinesis tests when the Kinesis assembly JAR has not been generated

2015-08-10 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-9640. -- Resolution: Fixed Fix Version/s: 1.5.0 > Do not run Python Kinesis tests when the Kinesis

[jira] [Updated] (SPARK-9809) Task crashes because the internal accumulators are not properly initialized

2015-08-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9809: --- Description: When a stage failed and another stage was resubmitted with only part of partitions to co

[jira] [Commented] (SPARK-9814) EqualNotNull not passing to data sources

2015-08-10 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681294#comment-14681294 ] Hyukjin Kwon commented on SPARK-9814: - I just made it :) > EqualNotNull not passing t

[jira] [Issue Comment Deleted] (SPARK-9814) EqualNotNull not passing to data sources

2015-08-10 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-9814: Comment: was deleted (was: I just made it. https://github.com/apache/spark/pull/8096) > EqualNotNul

[jira] [Commented] (SPARK-9814) EqualNotNull not passing to data sources

2015-08-10 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681293#comment-14681293 ] Hyukjin Kwon commented on SPARK-9814: - I just made it. https://github.com/apache/spark

[jira] [Assigned] (SPARK-9814) EqualNotNull not passing to data sources

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9814: --- Assignee: Apache Spark > EqualNotNull not passing to data sources > -

[jira] [Assigned] (SPARK-9814) EqualNotNull not passing to data sources

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9814: --- Assignee: (was: Apache Spark) > EqualNotNull not passing to data sources > --

[jira] [Commented] (SPARK-9814) EqualNotNull not passing to data sources

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681291#comment-14681291 ] Apache Spark commented on SPARK-9814: - User 'HyukjinKwon' has created a pull request f

[jira] [Commented] (SPARK-9814) EqualNotNull not passing to data sources

2015-08-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681289#comment-14681289 ] Reynold Xin commented on SPARK-9814: [~hyukjin.kwon] would you like to submit a patch

[jira] [Commented] (SPARK-7726) Maven Install Breaks When Upgrading Scala 2.11.2-->[2.11.3 or higher]

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681277#comment-14681277 ] Apache Spark commented on SPARK-7726: - User 'pwendell' has created a pull request for

[jira] [Updated] (SPARK-9814) EqualNotNull not passing to data sources

2015-08-10 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-9814: Component/s: (was: Input/Output) SQL > EqualNotNull not passing to data sources

[jira] [Updated] (SPARK-9814) EqualNotNull not passing to data sources

2015-08-10 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-9814: Environment: (was: Centos 6.6) > EqualNotNull not passing to data sources >

[jira] [Updated] (SPARK-9790) [YARN] Expose in WebUI if NodeManager is the reason why executors were killed.

2015-08-10 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated SPARK-9790: --- Attachment: error_showing_in_UI.png Attaching an image of what the error message in the UI would now l

[jira] [Comment Edited] (SPARK-9790) [YARN] Expose in WebUI if NodeManager is the reason why executors were killed.

2015-08-10 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681263#comment-14681263 ] Mark Grover edited comment on SPARK-9790 at 8/11/15 5:17 AM: -

[jira] [Resolved] (SPARK-9363) SortMergeJoin operator should support UnsafeRow

2015-08-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-9363. Resolution: Fixed Fix Version/s: 1.5.0 > SortMergeJoin operator should support UnsafeRow > --

[jira] [Resolved] (SPARK-9729) Sort Merge Join for Left and Right Outer Join

2015-08-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-9729. Resolution: Fixed Fix Version/s: 1.5.0 > Sort Merge Join for Left and Right Outer Join >

[jira] [Commented] (SPARK-9815) Rename PlatformDependent.UNSAFE -> Platform

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681250#comment-14681250 ] Apache Spark commented on SPARK-9815: - User 'rxin' has created a pull request for this

[jira] [Assigned] (SPARK-9815) Rename PlatformDependent.UNSAFE -> Platform

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9815: --- Assignee: Reynold Xin (was: Apache Spark) > Rename PlatformDependent.UNSAFE -> Platform > --

[jira] [Assigned] (SPARK-9815) Rename PlatformDependent.UNSAFE -> Platform

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9815: --- Assignee: Apache Spark (was: Reynold Xin) > Rename PlatformDependent.UNSAFE -> Platform > --

[jira] [Assigned] (SPARK-9790) [YARN] Expose in WebUI if NodeManager is the reason why executors were killed.

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9790: --- Assignee: Apache Spark > [YARN] Expose in WebUI if NodeManager is the reason why executors we

[jira] [Commented] (SPARK-9790) [YARN] Expose in WebUI if NodeManager is the reason why executors were killed.

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681249#comment-14681249 ] Apache Spark commented on SPARK-9790: - User 'markgrover' has created a pull request fo

[jira] [Assigned] (SPARK-9790) [YARN] Expose in WebUI if NodeManager is the reason why executors were killed.

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9790: --- Assignee: (was: Apache Spark) > [YARN] Expose in WebUI if NodeManager is the reason why e

[jira] [Updated] (SPARK-9814) EqualNotNull not passing to data sources

2015-08-10 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-9814: Description: When data sources (such as Parquet) tries to filter data when reading from HDFS (not i

[jira] [Created] (SPARK-9815) Rename PlatformDependent.UNSAFE -> Platform

2015-08-10 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-9815: -- Summary: Rename PlatformDependent.UNSAFE -> Platform Key: SPARK-9815 URL: https://issues.apache.org/jira/browse/SPARK-9815 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-9813) Incorrect UNION ALL behavior

2015-08-10 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681246#comment-14681246 ] Herman van Hovell commented on SPARK-9813: -- So I am not to sure if we want to mai

[jira] [Updated] (SPARK-9814) EqualNotNull not passing to data sources

2015-08-10 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-9814: Description: When data sources (such as Parquet) tries to filter data when reading from HDFS (not i

[jira] [Resolved] (SPARK-9340) CatalystSchemaConverter and CatalystRowConverter don't handle unannotated repeated fields correctly

2015-08-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-9340. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8070 [https://github.com/

[jira] [Created] (SPARK-9814) EqualNotNull not passing to data sources

2015-08-10 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-9814: --- Summary: EqualNotNull not passing to data sources Key: SPARK-9814 URL: https://issues.apache.org/jira/browse/SPARK-9814 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-8824) Support Parquet logical types TIMESTAMP_MILLIS and TIMESTAMP_MICROS

2015-08-10 Thread Konstantin Shaposhnikov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681146#comment-14681146 ] Konstantin Shaposhnikov commented on SPARK-8824: parquet-mr 1.7.0+ depends

[jira] [Updated] (SPARK-9813) Incorrect UNION ALL behavior

2015-08-10 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon Simeonov updated SPARK-9813: --- Description: According to the [Hive Language Manual|https://cwiki.apache.org/confluence/displ

[jira] [Updated] (SPARK-8115) Remove TestData

2015-08-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8115: - Assignee: Andrew Or > Remove TestData > --- > > Key: SPARK-8115 >

[jira] [Updated] (SPARK-8115) Remove TestData

2015-08-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8115: - Target Version/s: 1.5.0 > Remove TestData > --- > > Key: SPARK-8115 >

[jira] [Resolved] (SPARK-8114) Remove wildcard import on TestSQLContext._

2015-08-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-8114. -- Resolution: Fixed > Remove wildcard import on TestSQLContext._ > ---

[jira] [Updated] (SPARK-8109) TestSQLContext's static initialization is run during MiMa tests, causing SparkContexts to be created

2015-08-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8109: - Assignee: Andrew Or > TestSQLContext's static initialization is run during MiMa tests, causing > SparkCon

[jira] [Closed] (SPARK-8114) Remove wildcard import on TestSQLContext._

2015-08-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-8114. Resolution: Fixed Assignee: Reynold Xin Fix Version/s: 1.5.0 Target Version/s: 1

[jira] [Reopened] (SPARK-8114) Remove wildcard import on TestSQLContext._

2015-08-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reopened SPARK-8114: -- > Remove wildcard import on TestSQLContext._ > -- > >

[jira] [Updated] (SPARK-9580) Refactor TestSQLContext to make it non-singleton

2015-08-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-9580: - Issue Type: Sub-task (was: Bug) Parent: SPARK-8113 > Refactor TestSQLContext to make it non-singl

[jira] [Updated] (SPARK-8113) SQL module test cleanup

2015-08-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8113: - Priority: Major (was: Minor) > SQL module test cleanup > --- > > Key:

[jira] [Created] (SPARK-9813) Incorrect UNION ALL behavior

2015-08-10 Thread Simeon Simeonov (JIRA)
Simeon Simeonov created SPARK-9813: -- Summary: Incorrect UNION ALL behavior Key: SPARK-9813 URL: https://issues.apache.org/jira/browse/SPARK-9813 Project: Spark Issue Type: Bug Comp

[jira] [Commented] (SPARK-9727) Make the Kinesis project SBT name and consistent with other streaming projects

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681125#comment-14681125 ] Apache Spark commented on SPARK-9727: - User 'tdas' has created a pull request for this

[jira] [Created] (SPARK-9812) Test all new Python API examples

2015-08-10 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-9812: Summary: Test all new Python API examples Key: SPARK-9812 URL: https://issues.apache.org/jira/browse/SPARK-9812 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-9811) Test updated Kinesis Receiver

2015-08-10 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-9811: Summary: Test updated Kinesis Receiver Key: SPARK-9811 URL: https://issues.apache.org/jira/browse/SPARK-9811 Project: Spark Issue Type: Sub-task Co

[jira] [Assigned] (SPARK-9810) Remove individual commit messages from the squash commit message

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9810: --- Assignee: Reynold Xin (was: Apache Spark) > Remove individual commit messages from the squas

[jira] [Commented] (SPARK-9810) Remove individual commit messages from the squash commit message

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681104#comment-14681104 ] Apache Spark commented on SPARK-9810: - User 'rxin' has created a pull request for this

[jira] [Assigned] (SPARK-9810) Remove individual commit messages from the squash commit message

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9810: --- Assignee: Apache Spark (was: Reynold Xin) > Remove individual commit messages from the squas

[jira] [Commented] (SPARK-9809) Task crashes because the internal accumulators are not properly initialized

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681102#comment-14681102 ] Apache Spark commented on SPARK-9809: - User 'carsonwang' has created a pull request fo

[jira] [Created] (SPARK-9810) Remove individual commit messages from the squash commit message

2015-08-10 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-9810: -- Summary: Remove individual commit messages from the squash commit message Key: SPARK-9810 URL: https://issues.apache.org/jira/browse/SPARK-9810 Project: Spark I

[jira] [Updated] (SPARK-9809) Task crashes because the internal accumulators are not properly initialized

2015-08-10 Thread Carson Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carson Wang updated SPARK-9809: --- Description: When a stage failed and another stage was resubmitted with only part of partitions to co

[jira] [Assigned] (SPARK-9809) Task crashes because the internal accumulators are not properly initialized

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9809: --- Assignee: (was: Apache Spark) > Task crashes because the internal accumulators are not pr

[jira] [Assigned] (SPARK-9809) Task crashes because the internal accumulators are not properly initialized

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9809: --- Assignee: Apache Spark > Task crashes because the internal accumulators are not properly init

[jira] [Updated] (SPARK-9808) Remove hash shuffle file consolidation

2015-08-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9808: --- Target Version/s: 1.6.0 > Remove hash shuffle file consolidation > ---

[jira] [Created] (SPARK-9809) Task crashes because the internal accumulators are not properly initialized

2015-08-10 Thread Carson Wang (JIRA)
Carson Wang created SPARK-9809: -- Summary: Task crashes because the internal accumulators are not properly initialized Key: SPARK-9809 URL: https://issues.apache.org/jira/browse/SPARK-9809 Project: Spark

[jira] [Commented] (SPARK-9808) Remove hash shuffle file consolidation

2015-08-10 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681082#comment-14681082 ] Josh Rosen commented on SPARK-9808: --- Note that we can close https://issues.apache.org/ji

[jira] [Assigned] (SPARK-9808) Remove hash shuffle file consolidation

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9808: --- Assignee: Josh Rosen (was: Apache Spark) > Remove hash shuffle file consolidation >

[jira] [Commented] (SPARK-9808) Remove hash shuffle file consolidation

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681078#comment-14681078 ] Apache Spark commented on SPARK-9808: - User 'JoshRosen' has created a pull request for

[jira] [Assigned] (SPARK-9808) Remove hash shuffle file consolidation

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9808: --- Assignee: Apache Spark (was: Josh Rosen) > Remove hash shuffle file consolidation >

[jira] [Created] (SPARK-9808) Remove hash shuffle file consolidation

2015-08-10 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-9808: - Summary: Remove hash shuffle file consolidation Key: SPARK-9808 URL: https://issues.apache.org/jira/browse/SPARK-9808 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-9806) Don't share ReplayListenerBus between multiple applications

2015-08-10 Thread Rohit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Agarwal updated SPARK-9806: - Description: Currently, we are sharing {{ReplayListenerBus}} for replaying the event logs of vari

[jira] [Updated] (SPARK-8965) Add ml-guide Python Example: Estimator, Transformer, and Param

2015-08-10 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-8965: - Assignee: Rosstin Murphy > Add ml-guide Python Example: Estimator, Transformer, and Param

[jira] [Created] (SPARK-9807) pyspark.sql.createDataFrame does not infer data type of parsed TSV

2015-08-10 Thread Karen Yin-Yee Ng (JIRA)
Karen Yin-Yee Ng created SPARK-9807: --- Summary: pyspark.sql.createDataFrame does not infer data type of parsed TSV Key: SPARK-9807 URL: https://issues.apache.org/jira/browse/SPARK-9807 Project: Spark

[jira] [Commented] (SPARK-9806) Don't share ReplayListenerBus between multiple applications

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681030#comment-14681030 ] Apache Spark commented on SPARK-9806: - User 'mindprince' has created a pull request fo

[jira] [Assigned] (SPARK-9806) Don't share ReplayListenerBus between multiple applications

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9806: --- Assignee: Apache Spark > Don't share ReplayListenerBus between multiple applications > --

[jira] [Assigned] (SPARK-9806) Don't share ReplayListenerBus between multiple applications

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9806: --- Assignee: (was: Apache Spark) > Don't share ReplayListenerBus between multiple applicatio

[jira] [Created] (SPARK-9806) Don't share ReplayListenerBus between multiple applications

2015-08-10 Thread Rohit Agarwal (JIRA)
Rohit Agarwal created SPARK-9806: Summary: Don't share ReplayListenerBus between multiple applications Key: SPARK-9806 URL: https://issues.apache.org/jira/browse/SPARK-9806 Project: Spark Is

[jira] [Commented] (SPARK-9801) Spark streaming deletes the temp file and backup files without checking if they exist or not

2015-08-10 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681016#comment-14681016 ] Tathagata Das commented on SPARK-9801: -- I cant find it [~joshrosen] > Spark streamin

[jira] [Commented] (SPARK-9805) Make streaming PySpark ML tests more robust using termination conditions

2015-08-10 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680997#comment-14680997 ] Joseph K. Bradley commented on SPARK-9805: -- Example failures: * [https://amplab.

[jira] [Resolved] (SPARK-9801) Spark streaming deletes the temp file and backup files without checking if they exist or not

2015-08-10 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-9801. -- Resolution: Fixed Fix Version/s: 1.5.0 1.4.2 1.3.2

[jira] [Updated] (SPARK-9606) HiveThriftServer tests failing.

2015-08-10 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-9606: Sprint: Spark 1.5 release > HiveThriftServer tests failing. > --

[jira] [Updated] (SPARK-9757) Can't create persistent data source tables with decimal

2015-08-10 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-9757: Sprint: Spark 1.5 release > Can't create persistent data source tables with decimal > --

[jira] [Updated] (SPARK-9407) Parquet shouldn't push down predicates over a column whose underlying Parquet type is an ENUM

2015-08-10 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-9407: Sprint: Spark 1.5 release > Parquet shouldn't push down predicates over a column whose under

[jira] [Updated] (SPARK-9604) Unsafe ArrayData and MapData is very very slow

2015-08-10 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-9604: Target Version/s: 1.6.0 (was: 1.5.0) > Unsafe ArrayData and MapData is very very slow > ---

[jira] [Commented] (SPARK-9340) CatalystSchemaConverter and CatalystRowConverter don't handle unannotated repeated fields correctly

2015-08-10 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680978#comment-14680978 ] Ryan Blue commented on SPARK-9340: -- I just took a look at PR #8070 and it looks good to m

[jira] [Commented] (SPARK-9776) Another instance of Derby may have already booted the database

2015-08-10 Thread Sudhakar Thota (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680977#comment-14680977 ] Sudhakar Thota commented on SPARK-9776: --- Sean, I have requested my colleague Mike t

[jira] [Assigned] (SPARK-9805) Make streaming PySpark ML tests more robust using termination conditions

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9805: --- Assignee: Joseph K. Bradley (was: Apache Spark) > Make streaming PySpark ML tests more robus

[jira] [Commented] (SPARK-9805) Make streaming PySpark ML tests more robust using termination conditions

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680958#comment-14680958 ] Apache Spark commented on SPARK-9805: - User 'jkbradley' has created a pull request for

[jira] [Assigned] (SPARK-9805) Make streaming PySpark ML tests more robust using termination conditions

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9805: --- Assignee: Apache Spark (was: Joseph K. Bradley) > Make streaming PySpark ML tests more robus

[jira] [Resolved] (SPARK-5155) Python API for MQTT streaming

2015-08-10 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-5155. -- Resolution: Fixed Fix Version/s: 1.5.0 > Python API for MQTT streaming >

[jira] [Commented] (SPARK-9804) "isSrcLocal" parameter in loadTable / loadPartition is incorrect for HDFS source data

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680953#comment-14680953 ] Apache Spark commented on SPARK-9804: - User 'vanzin' has created a pull request for th

[jira] [Assigned] (SPARK-9804) "isSrcLocal" parameter in loadTable / loadPartition is incorrect for HDFS source data

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9804: --- Assignee: (was: Apache Spark) > "isSrcLocal" parameter in loadTable / loadPartition is in

[jira] [Assigned] (SPARK-9804) "isSrcLocal" parameter in loadTable / loadPartition is incorrect for HDFS source data

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9804: --- Assignee: Apache Spark > "isSrcLocal" parameter in loadTable / loadPartition is incorrect for

[jira] [Created] (SPARK-9805) Make streaming PySpark ML tests more robust using termination conditions

2015-08-10 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-9805: Summary: Make streaming PySpark ML tests more robust using termination conditions Key: SPARK-9805 URL: https://issues.apache.org/jira/browse/SPARK-9805 Projec

[jira] [Created] (SPARK-9804) "isSrcLocal" parameter in loadTable / loadPartition is incorrect for HDFS source data

2015-08-10 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-9804: - Summary: "isSrcLocal" parameter in loadTable / loadPartition is incorrect for HDFS source data Key: SPARK-9804 URL: https://issues.apache.org/jira/browse/SPARK-9804

[jira] [Commented] (SPARK-8757) Check missing and add user guide for MLlib Python API

2015-08-10 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680945#comment-14680945 ] Joseph K. Bradley commented on SPARK-8757: -- Some of those items do have sections

[jira] [Assigned] (SPARK-9713) Document SparkR MLlib glm() integration in Spark 1.5

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9713: --- Assignee: (was: Apache Spark) > Document SparkR MLlib glm() integration in Spark 1.5 > --

[jira] [Assigned] (SPARK-9713) Document SparkR MLlib glm() integration in Spark 1.5

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9713: --- Assignee: Apache Spark > Document SparkR MLlib glm() integration in Spark 1.5 > -

[jira] [Commented] (SPARK-9713) Document SparkR MLlib glm() integration in Spark 1.5

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680921#comment-14680921 ] Apache Spark commented on SPARK-9713: - User 'ericl' has created a pull request for thi

[jira] [Commented] (SPARK-9800) GradientDescent$.runMiniBatchSGD

2015-08-10 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680914#comment-14680914 ] Joseph K. Bradley commented on SPARK-9800: -- The 1st version's doc specifies the d

[jira] [Commented] (SPARK-9649) Flaky test: o.a.s.deploy.master.MasterSuite: recovery

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680906#comment-14680906 ] Apache Spark commented on SPARK-9649: - User 'andrewor14' has created a pull request fo

[jira] [Reopened] (SPARK-9649) Flaky test: o.a.s.deploy.master.MasterSuite: recovery

2015-08-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reopened SPARK-9649: -- Seeing this again in master branch https://amplab.cs.berkeley.edu/jenkins/job/Spark-Master-SBT/3207/AMPLAB_

[jira] [Commented] (SPARK-9702) Repartition operator should use Exchange to perform its shuffle

2015-08-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680894#comment-14680894 ] Apache Spark commented on SPARK-9702: - User 'JoshRosen' has created a pull request for

[jira] [Updated] (SPARK-9737) Add the suggested configuration when required executor memory is above the max threshold of this cluster on YARN mode

2015-08-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9737: --- Fix Version/s: 1.5.0 > Add the suggested configuration when required executor memory is above the > m

[jira] [Commented] (SPARK-7726) Maven Install Breaks When Upgrading Scala 2.11.2-->[2.11.3 or higher]

2015-08-10 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680885#comment-14680885 ] Patrick Wendell commented on SPARK-7726: [~srowen] [~dragos] This is cropping up a

[jira] [Commented] (SPARK-9801) Spark streaming deletes the temp file and backup files without checking if they exist or not

2015-08-10 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680879#comment-14680879 ] Josh Rosen commented on SPARK-9801: --- I think there's an earlier duplicate JIRA for this

[jira] [Created] (SPARK-9803) Add transform and subset to DataFrame

2015-08-10 Thread Hossein Falaki (JIRA)
Hossein Falaki created SPARK-9803: - Summary: Add transform and subset to DataFrame Key: SPARK-9803 URL: https://issues.apache.org/jira/browse/SPARK-9803 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-9801) Spark streaming deletes the temp file and backup files without checking if they exist or not

2015-08-10 Thread Hao Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680843#comment-14680843 ] Hao Zhu commented on SPARK-9801: https://github.com/apache/spark/pull/8082 > Spark stream

  1   2   3   >