[jira] [Commented] (SPARK-15861) pyspark mapPartitions with none generator functions / functors

2016-06-15 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331975#comment-15331975 ] Bryan Cutler commented on SPARK-15861: -- If you change your function to this {noformat} def

[jira] [Updated] (SPARK-15965) No FileSystem for scheme: s3n or s3a spark-2.0.0 and spark-1.6.1

2016-06-15 Thread thauvin damien (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] thauvin damien updated SPARK-15965: --- Description: The spark programming-guide explain that Spark can create distributed datasets

[jira] [Commented] (SPARK-15849) FileNotFoundException on _temporary while doing saveAsTable to S3

2016-06-15 Thread Sandeep (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332034#comment-15332034 ] Sandeep commented on SPARK-15849: - Thanks Thomas for that comment. I have verified both the things with

[jira] [Commented] (SPARK-15815) Hang while enable blacklistExecutor and DynamicExecutorAllocator

2016-06-15 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331907#comment-15331907 ] Imran Rashid commented on SPARK-15815: -- yes, I agree that SPARK-15865 isn't ideal on its own.

[jira] [Resolved] (SPARK-15426) Spark 2.0 SQL API audit

2016-06-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15426. - Resolution: Fixed Fix Version/s: 2.0.0 > Spark 2.0 SQL API audit >

[jira] [Updated] (SPARK-15960) Audit new SQL confs

2016-06-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-15960: Issue Type: Sub-task (was: Bug) Parent: SPARK-15426 > Audit new SQL confs >

[jira] [Resolved] (SPARK-15960) Audit new SQL confs

2016-06-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15960. - Resolution: Fixed Fix Version/s: 2.0.0 > Audit new SQL confs > > >

[jira] [Updated] (SPARK-15951) Change Executors Page to use datatables to support sorting columns and searching

2016-06-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15951: -- Priority: Minor (was: Major) Fix Version/s: (was: 2.1.0) Component/s: (was:

[jira] [Comment Edited] (SPARK-15908) Add varargs-type dropDuplicates() function in SparkR

2016-06-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331439#comment-15331439 ] Dongjoon Hyun edited comment on SPARK-15908 at 6/15/16 10:16 AM: - I'll

[jira] [Commented] (SPARK-15908) Add varargs-type dropDuplicates() function in SparkR

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331488#comment-15331488 ] Apache Spark commented on SPARK-15908: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-15908) Add varargs-type dropDuplicates() function in SparkR

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15908: Assignee: Apache Spark > Add varargs-type dropDuplicates() function in SparkR >

[jira] [Assigned] (SPARK-15908) Add varargs-type dropDuplicates() function in SparkR

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15908: Assignee: (was: Apache Spark) > Add varargs-type dropDuplicates() function in SparkR

[jira] [Updated] (SPARK-15065) HiveSparkSubmitSuite's "set spark.sql.warehouse.dir" is flaky

2016-06-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15065: -- Assignee: Pete Robbins > HiveSparkSubmitSuite's "set spark.sql.warehouse.dir" is flaky >

[jira] [Updated] (SPARK-15631) Dataset and encoder bug fixes

2016-06-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15631: -- Assignee: Wenchen Fan > Dataset and encoder bug fixes > - > >

[jira] [Commented] (SPARK-15908) Add varargs-type dropDuplicates() function in SparkR

2016-06-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331439#comment-15331439 ] Dongjoon Hyun commented on SPARK-15908: --- I'll working on this issue~. > Add varargs-type

[jira] [Commented] (SPARK-15518) Rename various scheduler backend for consistency

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331455#comment-15331455 ] Apache Spark commented on SPARK-15518: -- User 'lw-lin' has created a pull request for this issue:

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-06-15 Thread Pete Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331370#comment-15331370 ] Pete Robbins commented on SPARK-15822: -- Chatting with [~hvanhovell] here is the current state. I can

[jira] [Assigned] (SPARK-15960) Audit new SQL confs

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15960: Assignee: Herman van Hovell (was: Apache Spark) > Audit new SQL confs >

[jira] [Assigned] (SPARK-15960) Audit new SQL confs

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15960: Assignee: Apache Spark (was: Herman van Hovell) > Audit new SQL confs >

[jira] [Commented] (SPARK-15960) Audit new SQL confs

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331244#comment-15331244 ] Apache Spark commented on SPARK-15960: -- User 'hvanhovell' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-15815) Hang while enable blacklistExecutor and DynamicExecutorAllocator

2016-06-15 Thread SuYan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331285#comment-15331285 ] SuYan edited comment on SPARK-15815 at 6/15/16 7:17 AM: I see... although it can

[jira] [Commented] (SPARK-15815) Hang while enable blacklistExecutor and DynamicExecutorAllocator

2016-06-15 Thread SuYan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331285#comment-15331285 ] SuYan commented on SPARK-15815: --- I see... although it can solve the gang problem, but for Dynamic Allocate,

[jira] [Created] (SPARK-15962) Introduce additonal implementation with a dense format for UnsafeArrayData

2016-06-15 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-15962: Summary: Introduce additonal implementation with a dense format for UnsafeArrayData Key: SPARK-15962 URL: https://issues.apache.org/jira/browse/SPARK-15962

[jira] [Resolved] (SPARK-15397) 'locate' UDF got different result with boundary value case compared to Hive engine

2016-06-15 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang resolved SPARK-15397. - Resolution: Fixed > 'locate' UDF got different result with boundary value case compared to Hive

[jira] [Assigned] (SPARK-15888) Python UDF over aggregate fails

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15888: Assignee: Apache Spark (was: Davies Liu) > Python UDF over aggregate fails >

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-06-15 Thread Pete Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331371#comment-15331371 ] Pete Robbins commented on SPARK-15822: -- The generated code is: {code} Top Arrival Carrier

[jira] [Commented] (SPARK-15959) Add the support of hive.metastore.warehouse.dir back

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331206#comment-15331206 ] Apache Spark commented on SPARK-15959: -- User 'yhuai' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15959) Add the support of hive.metastore.warehouse.dir back

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15959: Assignee: Yin Huai (was: Apache Spark) > Add the support of hive.metastore.warehouse.dir

[jira] [Assigned] (SPARK-15959) Add the support of hive.metastore.warehouse.dir back

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15959: Assignee: Apache Spark (was: Yin Huai) > Add the support of hive.metastore.warehouse.dir

[jira] [Commented] (SPARK-15962) Introduce additonal implementation with a dense format for UnsafeArrayData

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331222#comment-15331222 ] Apache Spark commented on SPARK-15962: -- User 'kiszk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15962) Introduce additonal implementation with a dense format for UnsafeArrayData

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15962: Assignee: (was: Apache Spark) > Introduce additonal implementation with a dense

[jira] [Assigned] (SPARK-15962) Introduce additonal implementation with a dense format for UnsafeArrayData

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15962: Assignee: Apache Spark > Introduce additonal implementation with a dense format for

[jira] [Commented] (SPARK-15955) Failed Spark application returns with exitcode equals to zero

2016-06-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331325#comment-15331325 ] Sean Owen commented on SPARK-15955: --- Which process has status 0, and can you try vs master? > Failed

[jira] [Commented] (SPARK-15888) Python UDF over aggregate fails

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331266#comment-15331266 ] Apache Spark commented on SPARK-15888: -- User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15888) Python UDF over aggregate fails

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15888: Assignee: Davies Liu (was: Apache Spark) > Python UDF over aggregate fails >

[jira] [Commented] (SPARK-15922) BlockMatrix to IndexedRowMatrix throws an error

2016-06-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331294#comment-15331294 ] Dongjoon Hyun commented on SPARK-15922: --- Hi, [~mengxr]. Could you review the PR on this bug? >

[jira] [Updated] (SPARK-15888) Python UDF over aggregate fails

2016-06-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-15888: Affects Version/s: (was: 2.0.0) > Python UDF over aggregate fails >

[jira] [Commented] (SPARK-1004) PySpark on YARN

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331314#comment-15331314 ] Apache Spark commented on SPARK-1004: - User 'sryza' has created a pull request for this issue:

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-06-15 Thread Pete Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331375#comment-15331375 ] Pete Robbins commented on SPARK-15822: -- and the plan: {noformat} == Parsed Logical Plan == 'Project

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-06-15 Thread Pete Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331580#comment-15331580 ] Pete Robbins commented on SPARK-15822: -- I can also recreate this issue on Oracle JDK 1.8:

[jira] [Commented] (SPARK-14692) Error While Setting the path for R front end

2016-06-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331613#comment-15331613 ] Dongjoon Hyun commented on SPARK-14692: --- Hi, [~nmolkeri]. It seems that you just needed

[jira] [Created] (SPARK-15965) No FileSystem for scheme: s3n or s3a spark-2.0.0 and spark-1.6.1

2016-06-15 Thread thauvin damien (JIRA)
thauvin damien created SPARK-15965: -- Summary: No FileSystem for scheme: s3n or s3a spark-2.0.0 and spark-1.6.1 Key: SPARK-15965 URL: https://issues.apache.org/jira/browse/SPARK-15965 Project: Spark

[jira] [Created] (SPARK-15966) Fix markdown for Spark Monitoring

2016-06-15 Thread Dhruve Ashar (JIRA)
Dhruve Ashar created SPARK-15966: Summary: Fix markdown for Spark Monitoring Key: SPARK-15966 URL: https://issues.apache.org/jira/browse/SPARK-15966 Project: Spark Issue Type: Documentation

[jira] [Updated] (SPARK-15046) When running hive-thriftserver with yarn on a secure cluster the workers fail with java.lang.NumberFormatException

2016-06-15 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-15046: -- Assignee: Marcelo Vanzin > When running hive-thriftserver with yarn on a secure cluster the

[jira] [Resolved] (SPARK-15046) When running hive-thriftserver with yarn on a secure cluster the workers fail with java.lang.NumberFormatException

2016-06-15 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-15046. --- Resolution: Fixed Fix Version/s: 2.0.0 > When running hive-thriftserver with yarn on

[jira] [Created] (SPARK-15967) Spark UI should show dynamically changed value of storage memory instead of showing one static value all the time

2016-06-15 Thread Umesh K (JIRA)
Umesh K created SPARK-15967: --- Summary: Spark UI should show dynamically changed value of storage memory instead of showing one static value all the time Key: SPARK-15967 URL:

[jira] [Commented] (SPARK-15968) HiveMetastoreCatalog does not correctly validate partitioned metastore relation when searching the internal table cache

2016-06-15 Thread Michael Allman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332074#comment-15332074 ] Michael Allman commented on SPARK-15968: I have a patched based on the way this method was

[jira] [Created] (SPARK-15968) HiveMetastoreCatalog does not correctly validate partitioned metastore relation when searching the internal table cache

2016-06-15 Thread Michael Allman (JIRA)
Michael Allman created SPARK-15968: -- Summary: HiveMetastoreCatalog does not correctly validate partitioned metastore relation when searching the internal table cache Key: SPARK-15968 URL:

[jira] [Updated] (SPARK-15967) Spark UI should show realtime value of storage memory instead of showing one static value all the time

2016-06-15 Thread Umesh K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umesh K updated SPARK-15967: Summary: Spark UI should show realtime value of storage memory instead of showing one static value all the

[jira] [Updated] (SPARK-15968) HiveMetastoreCatalog does not correctly validate partitioned metastore relation when searching the internal table cache

2016-06-15 Thread Michael Allman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Allman updated SPARK-15968: --- Description: The {{getCached}} method of {{HiveMetastoreCatalog}} computes

[jira] [Commented] (SPARK-15968) HiveMetastoreCatalog does not correctly validate partitioned metastore relation when searching the internal table cache

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332123#comment-15332123 ] Apache Spark commented on SPARK-15968: -- User 'mallman' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15968) HiveMetastoreCatalog does not correctly validate partitioned metastore relation when searching the internal table cache

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15968: Assignee: (was: Apache Spark) > HiveMetastoreCatalog does not correctly validate

[jira] [Assigned] (SPARK-15968) HiveMetastoreCatalog does not correctly validate partitioned metastore relation when searching the internal table cache

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15968: Assignee: Apache Spark > HiveMetastoreCatalog does not correctly validate partitioned

[jira] [Updated] (SPARK-15811) UDFs do not work in Spark 2.0-preview built with scala 2.10

2016-06-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15811: --- Priority: Blocker (was: Critical) > UDFs do not work in Spark 2.0-preview built with scala 2.10 >

[jira] [Resolved] (SPARK-15931) SparkR tests failing on R 3.3.0

2016-06-15 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-15931. --- Resolution: Fixed Assignee: Felix Cheung Fix Version/s: 2.0.0

[jira] [Assigned] (SPARK-15811) UDFs do not work in Spark 2.0-preview built with scala 2.10

2016-06-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-15811: -- Assignee: Davies Liu > UDFs do not work in Spark 2.0-preview built with scala 2.10 >

[jira] [Resolved] (SPARK-15953) Renamed ContinuousQuery to StreamingQuery for simplicity

2016-06-15 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-15953. -- Resolution: Fixed Fix Version/s: 2.0.0 > Renamed ContinuousQuery to StreamingQuery for

[jira] [Commented] (SPARK-15784) Add Power Iteration Clustering to spark.ml

2016-06-15 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332185#comment-15332185 ] Miao Wang commented on SPARK-15784: --- [~josephkb][~mengxr][~yanboliang] I am trying to add PIC to

[jira] [Created] (SPARK-15963) `TaskKilledException` is not correctly caught in `Executor.TaskRunner`

2016-06-15 Thread Liwei Lin (JIRA)
Liwei Lin created SPARK-15963: - Summary: `TaskKilledException` is not correctly caught in `Executor.TaskRunner` Key: SPARK-15963 URL: https://issues.apache.org/jira/browse/SPARK-15963 Project: Spark

[jira] [Assigned] (SPARK-15963) `TaskKilledException` is not correctly caught in `Executor.TaskRunner`

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15963: Assignee: Apache Spark > `TaskKilledException` is not correctly caught in

[jira] [Assigned] (SPARK-15963) `TaskKilledException` is not correctly caught in `Executor.TaskRunner`

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15963: Assignee: (was: Apache Spark) > `TaskKilledException` is not correctly caught in

[jira] [Commented] (SPARK-15963) `TaskKilledException` is not correctly caught in `Executor.TaskRunner`

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331837#comment-15331837 ] Apache Spark commented on SPARK-15963: -- User 'lw-lin' has created a pull request for this issue:

[jira] [Created] (SPARK-15964) Assignment to RDD-typed val fails

2016-06-15 Thread Sanjay Dasgupta (JIRA)
Sanjay Dasgupta created SPARK-15964: --- Summary: Assignment to RDD-typed val fails Key: SPARK-15964 URL: https://issues.apache.org/jira/browse/SPARK-15964 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-15969) FileNotFoundException: Multiple arguments for py-files flag, (also jars) for spark-submit

2016-06-15 Thread Kun Liu (JIRA)
Kun Liu created SPARK-15969: --- Summary: FileNotFoundException: Multiple arguments for py-files flag, (also jars) for spark-submit Key: SPARK-15969 URL: https://issues.apache.org/jira/browse/SPARK-15969

[jira] [Resolved] (SPARK-15826) PipedRDD to allow configurable char encoding

2016-06-15 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-15826. -- Resolution: Fixed Assignee: Tejas Patil Fix Version/s: 2.0.0 > PipedRDD to

[jira] [Commented] (SPARK-15973) GroupedData.pivot documentation off

2016-06-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332364#comment-15332364 ] Sean Owen commented on SPARK-15973: --- Please group your 3 JIRAs into one. They sound so similar that

[jira] [Resolved] (SPARK-15959) Add the support of hive.metastore.warehouse.dir back

2016-06-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15959. - Resolution: Fixed Fix Version/s: 2.0.0 > Add the support of hive.metastore.warehouse.dir

[jira] [Created] (SPARK-15971) GroupedData's member incorrectly named

2016-06-15 Thread Vladimir Feinberg (JIRA)
Vladimir Feinberg created SPARK-15971: - Summary: GroupedData's member incorrectly named Key: SPARK-15971 URL: https://issues.apache.org/jira/browse/SPARK-15971 Project: Spark Issue Type:

[jira] [Created] (SPARK-15973) GroupedData.pivot documentation off

2016-06-15 Thread Vladimir Feinberg (JIRA)
Vladimir Feinberg created SPARK-15973: - Summary: GroupedData.pivot documentation off Key: SPARK-15973 URL: https://issues.apache.org/jira/browse/SPARK-15973 Project: Spark Issue Type:

[jira] [Commented] (SPARK-15966) Fix markdown for Spark Monitoring

2016-06-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332371#comment-15332371 ] Sean Owen commented on SPARK-15966: --- Please make a more descriptive JIRA, and/or submit a PR. > Fix

[jira] [Updated] (SPARK-15973) Fix GroupedData Documentation

2016-06-15 Thread Vladimir Feinberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Feinberg updated SPARK-15973: -- Description: (1) {{GroupedData.pivot}} documenation uses {{//}} instead of {{#}} for

[jira] [Commented] (SPARK-15955) Failed Spark application returns with exitcode equals to zero

2016-06-15 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332276#comment-15332276 ] Thomas Graves commented on SPARK-15955: --- what master and deploy mode are you using? > Failed Spark

[jira] [Resolved] (SPARK-13850) TimSort Comparison method violates its general contract

2016-06-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-13850. - Resolution: Fixed Fix Version/s: 2.0.0 1.6.2 > TimSort Comparison

[jira] [Updated] (SPARK-15826) PipedRDD to allow configurable char encoding

2016-06-15 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tejas Patil updated SPARK-15826: Summary: PipedRDD to allow configurable char encoding (was: PipedRDD to allow configurable char

[jira] [Commented] (SPARK-15965) No FileSystem for scheme: s3n or s3a spark-2.0.0 and spark-1.6.1

2016-06-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332369#comment-15332369 ] Sean Owen commented on SPARK-15965: --- CC [~steve_l] but I think this is your classpath issue or a Hadoop

[jira] [Updated] (SPARK-15973) Fix GroupedData Documentation

2016-06-15 Thread Vladimir Feinberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Feinberg updated SPARK-15973: -- Summary: Fix GroupedData Documentation (was: GroupedData.pivot documentation off) >

[jira] [Created] (SPARK-15970) WARNing message related to persisting table to Hive Megastore while Spark SQL is running in-memory catalog mode

2016-06-15 Thread Xin Wu (JIRA)
Xin Wu created SPARK-15970: -- Summary: WARNing message related to persisting table to Hive Megastore while Spark SQL is running in-memory catalog mode Key: SPARK-15970 URL:

[jira] [Commented] (SPARK-15970) WARNing message related to persisting table to Hive Megastore while Spark SQL is running in-memory catalog mode

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332331#comment-15332331 ] Apache Spark commented on SPARK-15970: -- User 'xwu0226' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15970) WARNing message related to persisting table to Hive Megastore while Spark SQL is running in-memory catalog mode

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15970: Assignee: (was: Apache Spark) > WARNing message related to persisting table to Hive

[jira] [Updated] (SPARK-15971) GroupedData's member incorrectly named

2016-06-15 Thread Vladimir Feinberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Feinberg updated SPARK-15971: -- Description: The {{pyspark.sql.GroupedData}} object calls the Java object it wraps

[jira] [Updated] (SPARK-15972) GroupedData varargs arguments misnamed

2016-06-15 Thread Vladimir Feinberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Feinberg updated SPARK-15972: -- Description: Simple aggregation functions which take column names {{cols}} as varargs

[jira] [Resolved] (SPARK-15964) Assignment to RDD-typed val fails

2016-06-15 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-15964. -- Resolution: Won't Fix > Assignment to RDD-typed val fails >

[jira] [Commented] (SPARK-15964) Assignment to RDD-typed val fails

2016-06-15 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332224#comment-15332224 ] Michael Armbrust commented on SPARK-15964: -- Thanks for reporting this, but I believe this is

[jira] [Assigned] (SPARK-15947) Make pipeline components backward compatible with old vector columns in Scala/Java

2016-06-15 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-15947: - Assignee: Xiangrui Meng > Make pipeline components backward compatible with old vector

[jira] [Commented] (SPARK-15973) Fix GroupedData Documentation

2016-06-15 Thread Vladimir Feinberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332374#comment-15332374 ] Vladimir Feinberg commented on SPARK-15973: --- Done > Fix GroupedData Documentation >

[jira] [Commented] (SPARK-3847) Enum.hashCode is only consistent within the same JVM

2016-06-15 Thread Brett Stime (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332453#comment-15332453 ] Brett Stime commented on SPARK-3847: Seems like, rather than a warning about the specifics of enums,

[jira] [Updated] (SPARK-15969) FileNotFoundException: Multiple arguments for py-files flag, (also jars) for spark-submit

2016-06-15 Thread Kun Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kun Liu updated SPARK-15969: Remaining Estimate: 120h (was: 168h) Original Estimate: 120h (was: 168h) > FileNotFoundException:

[jira] [Updated] (SPARK-15811) Python UDFs do not work in Spark 2.0-preview built with scala 2.10

2016-06-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15811: --- Summary: Python UDFs do not work in Spark 2.0-preview built with scala 2.10 (was: UDFs do not work

[jira] [Assigned] (SPARK-15970) WARNing message related to persisting table to Hive Megastore while Spark SQL is running in-memory catalog mode

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15970: Assignee: Apache Spark > WARNing message related to persisting table to Hive Megastore

[jira] [Resolved] (SPARK-15971) GroupedData's member incorrectly named

2016-06-15 Thread Vladimir Feinberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Feinberg resolved SPARK-15971. --- Resolution: Duplicate > GroupedData's member incorrectly named >

[jira] [Resolved] (SPARK-15972) GroupedData varargs arguments misnamed

2016-06-15 Thread Vladimir Feinberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Feinberg resolved SPARK-15972. --- Resolution: Duplicate > GroupedData varargs arguments misnamed >

[jira] [Closed] (SPARK-15972) GroupedData varargs arguments misnamed

2016-06-15 Thread Vladimir Feinberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Feinberg closed SPARK-15972. - > GroupedData varargs arguments misnamed > -- > >

[jira] [Closed] (SPARK-15971) GroupedData's member incorrectly named

2016-06-15 Thread Vladimir Feinberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Feinberg closed SPARK-15971. - > GroupedData's member incorrectly named > -- > >

[jira] [Updated] (SPARK-15811) Python UDFs do not work in Spark 2.0-preview built with scala 2.10

2016-06-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15811: --- Description: I've built spark-2.0-preview (8f5a04b) with scala-2.10 using the following {code}

[jira] [Resolved] (SPARK-15935) Enable test for sql/streaming.py and fix these tests

2016-06-15 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-15935. -- Resolution: Fixed Fix Version/s: 2.0.0 > Enable test for sql/streaming.py and fix these

[jira] [Created] (SPARK-15972) GroupedData varargs arguments misnamed

2016-06-15 Thread Vladimir Feinberg (JIRA)
Vladimir Feinberg created SPARK-15972: - Summary: GroupedData varargs arguments misnamed Key: SPARK-15972 URL: https://issues.apache.org/jira/browse/SPARK-15972 Project: Spark Issue Type:

[jira] [Commented] (SPARK-15574) Python meta-algorithms in Scala

2016-06-15 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332475#comment-15332475 ] Xusen Yin commented on SPARK-15574: --- I just finished the prototype of PythonTransformer in Scala as the

[jira] [Commented] (SPARK-1051) On Yarn, executors don't doAs as submitting user

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332539#comment-15332539 ] Apache Spark commented on SPARK-1051: - User 'sryza' has created a pull request for this issue:

[jira] [Updated] (SPARK-15906) Complementary Naive Bayes Algorithm Implementation

2016-06-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15906: -- Issue Type: Improvement (was: New Feature) > Complementary Naive Bayes Algorithm

[jira] [Updated] (SPARK-15906) Complementary Naive Bayes Algorithm Implementation

2016-06-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15906: -- Priority: Minor (was: Major) > Complementary Naive Bayes Algorithm Implementation >

  1   2   3   >