[jira] [Updated] (SPARK-15463) Support for creating a dataframe from CSV in Dataset[String]

2016-05-25 Thread PJ Fanning (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning updated SPARK-15463: --- Summary: Support for creating a dataframe from CSV in Dataset[String] (was: Support for creating a d

[jira] [Commented] (SPARK-15554) Duplicated executors in Spark UI

2016-05-25 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301627#comment-15301627 ] Davies Liu commented on SPARK-15554: cc [~zsxwing] > Duplicated executors in Spark U

[jira] [Created] (SPARK-15554) Duplicated executors in Spark UI

2016-05-25 Thread Davies Liu (JIRA)
Davies Liu created SPARK-15554: -- Summary: Duplicated executors in Spark UI Key: SPARK-15554 URL: https://issues.apache.org/jira/browse/SPARK-15554 Project: Spark Issue Type: Bug Compon

[jira] [Commented] (SPARK-15549) Bucket column only need to find in relation when use bucketed table

2016-05-25 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301625#comment-15301625 ] Takeshi Yamamuro commented on SPARK-15549: -- This seems to be a bug, so `Type` sh

[jira] [Created] (SPARK-15553) Dataset.createTempView should use CreateViewCommand

2016-05-25 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-15553: --- Summary: Dataset.createTempView should use CreateViewCommand Key: SPARK-15553 URL: https://issues.apache.org/jira/browse/SPARK-15553 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-15552) Remove unnecessary private[sql] methods in SparkSession

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15552: Assignee: Apache Spark (was: Reynold Xin) > Remove unnecessary private[sql] methods in Sp

[jira] [Assigned] (SPARK-15552) Remove unnecessary private[sql] methods in SparkSession

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15552: Assignee: Reynold Xin (was: Apache Spark) > Remove unnecessary private[sql] methods in Sp

[jira] [Commented] (SPARK-15552) Remove unnecessary private[sql] methods in SparkSession

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301620#comment-15301620 ] Apache Spark commented on SPARK-15552: -- User 'rxin' has created a pull request for t

[jira] [Assigned] (SPARK-15391) Spark executor OOM during TimSort

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15391: Assignee: Davies Liu (was: Apache Spark) > Spark executor OOM during TimSort > --

[jira] [Assigned] (SPARK-15391) Spark executor OOM during TimSort

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15391: Assignee: Apache Spark (was: Davies Liu) > Spark executor OOM during TimSort > --

[jira] [Commented] (SPARK-15391) Spark executor OOM during TimSort

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301616#comment-15301616 ] Apache Spark commented on SPARK-15391: -- User 'davies' has created a pull request for

[jira] [Created] (SPARK-15552) Remove unnecessary private[sql] methods in SparkSession

2016-05-25 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-15552: --- Summary: Remove unnecessary private[sql] methods in SparkSession Key: SPARK-15552 URL: https://issues.apache.org/jira/browse/SPARK-15552 Project: Spark Issue T

[jira] [Assigned] (SPARK-15551) Scaladoc for KeyValueGroupedDataset points to old method

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15551: Assignee: Apache Spark > Scaladoc for KeyValueGroupedDataset points to old method > --

[jira] [Assigned] (SPARK-15551) Scaladoc for KeyValueGroupedDataset points to old method

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15551: Assignee: (was: Apache Spark) > Scaladoc for KeyValueGroupedDataset points to old meth

[jira] [Commented] (SPARK-15551) Scaladoc for KeyValueGroupedDataset points to old method

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301611#comment-15301611 ] Apache Spark commented on SPARK-15551: -- User 'holdenk' has created a pull request fo

[jira] [Created] (SPARK-15551) Scaladoc for KeyValueGroupedDataset points to old method

2016-05-25 Thread holdenk (JIRA)
holdenk created SPARK-15551: --- Summary: Scaladoc for KeyValueGroupedDataset points to old method Key: SPARK-15551 URL: https://issues.apache.org/jira/browse/SPARK-15551 Project: Spark Issue Type: Do

[jira] [Commented] (SPARK-15541) SparkContext.stop throws error

2016-05-25 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301607#comment-15301607 ] Miao Wang commented on SPARK-15541: --- [~srowen][~jerryshao]Thanks for your explanation!

[jira] [Updated] (SPARK-15549) Bucket column only need to find in relation when use bucketed table

2016-05-25 Thread Yadong Qi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yadong Qi updated SPARK-15549: -- Description: I create a bucketed table test(i int, j int, k int) with bucket column i, {code:java} cas

[jira] [Updated] (SPARK-15550) Dataset.show() doesn't disply inner nested structs properly

2016-05-25 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15550: --- Description: Say we have the following nested case class: {code} case class ClassData(a: String, b:

[jira] [Updated] (SPARK-15547) Encoder validation is too strict for inner nested structs

2016-05-25 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15547: --- Description: The following Spark shell snippet reproduces this issue: {code} case class ClassData(a:

[jira] [Updated] (SPARK-15550) Dataset.show() doesn't disply inner nested structs properly

2016-05-25 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15550: --- Description: The following Spark shell snippet reproduces this issue: {code} case class ClassData(a:

[jira] [Created] (SPARK-15550) Dataset.show() doesn't disply inner nested structs properly

2016-05-25 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-15550: -- Summary: Dataset.show() doesn't disply inner nested structs properly Key: SPARK-15550 URL: https://issues.apache.org/jira/browse/SPARK-15550 Project: Spark Issu

[jira] [Updated] (SPARK-15549) Bucket column only need to find in relation when use bucketed table

2016-05-25 Thread Yadong Qi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yadong Qi updated SPARK-15549: -- Description: I create a bucketed table test(i int, j int, k int) with bucket column i, {code:scala} ca

[jira] [Commented] (SPARK-15506) only one notebook can define a UDF; java.sql.SQLException: Another instance of Derby may have already booted the database

2016-05-25 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301594#comment-15301594 ] Jeff Zhang commented on SPARK-15506: Do you use zeppelin or jupyter ? It seems each y

[jira] [Updated] (SPARK-15549) Bucket column only need to find in relation when use bucketed table

2016-05-25 Thread Yadong Qi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yadong Qi updated SPARK-15549: -- Description: I create a bucketed table test(i int, j int, k int) with bucket column i, {code:scala} ca

[jira] [Created] (SPARK-15549) Bucket column only need to find in relation when use bucketed table

2016-05-25 Thread Yadong Qi (JIRA)
Yadong Qi created SPARK-15549: - Summary: Bucket column only need to find in relation when use bucketed table Key: SPARK-15549 URL: https://issues.apache.org/jira/browse/SPARK-15549 Project: Spark

[jira] [Updated] (SPARK-15549) Bucket column only need to find in relation when use bucketed table

2016-05-25 Thread Yadong Qi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yadong Qi updated SPARK-15549: -- Description: I create a bucketed table test(i int, j int, k int) with bucket column i, {code:scala} ca

[jira] [Created] (SPARK-15548) Shuffle file remains with dynamic allocation on YARN

2016-05-25 Thread He Tianyi (JIRA)
He Tianyi created SPARK-15548: - Summary: Shuffle file remains with dynamic allocation on YARN Key: SPARK-15548 URL: https://issues.apache.org/jira/browse/SPARK-15548 Project: Spark Issue Type: Bu

[jira] [Updated] (SPARK-15547) Encoder validation is too strict for inner nested structs

2016-05-25 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15547: --- Description: The following Spark shell snippet reproduces this issue: {code} case class ClassData(a:

[jira] [Updated] (SPARK-15547) Encoder validation is too strict for inner nested structs

2016-05-25 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15547: --- Description: The following Spark shell snippet reproduces this issue: {code} case class ClassData(a:

[jira] [Created] (SPARK-15547) Encoder validation is too strict for inner nested structs

2016-05-25 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-15547: -- Summary: Encoder validation is too strict for inner nested structs Key: SPARK-15547 URL: https://issues.apache.org/jira/browse/SPARK-15547 Project: Spark Issue T

[jira] [Commented] (SPARK-12550) sbt-launch-lib.bash: line 72: 2404 Killed "$@"

2016-05-25 Thread Pete Clapham (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301569#comment-15301569 ] Pete Clapham commented on SPARK-12550: -- Thanks for your reply Sean,I followed the on

[jira] [Resolved] (SPARK-10372) Add end-to-end tests for the scheduling code

2016-05-25 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-10372. -- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 8559 [https://gith

[jira] [Issue Comment Deleted] (SPARK-5682) Add encrypted shuffle in spark

2016-05-25 Thread hujiayin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hujiayin updated SPARK-5682: Comment: was deleted (was: Steps were added to encode and decode the data, the performance will not be fast

[jira] [Issue Comment Deleted] (SPARK-5682) Add encrypted shuffle in spark

2016-05-25 Thread hujiayin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hujiayin updated SPARK-5682: Comment: was deleted (was: The solution relied on hadoop API and maybe downgrade the performance.  The AES a

[jira] [Issue Comment Deleted] (SPARK-5682) Add encrypted shuffle in spark

2016-05-25 Thread hujiayin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hujiayin updated SPARK-5682: Comment: was deleted (was: Since the encrypted shuffle in spark is focus on the common module, it maybe not

[jira] [Commented] (SPARK-15034) Use the value of spark.sql.warehouse.dir as the warehouse location instead of using hive.metastore.warehouse.dir

2016-05-25 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301528#comment-15301528 ] Yin Huai commented on SPARK-15034: -- Seems it is better to explicitly set local filesyste

[jira] [Commented] (SPARK-15034) Use the value of spark.sql.warehouse.dir as the warehouse location instead of using hive.metastore.warehouse.dir

2016-05-25 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301518#comment-15301518 ] Yin Huai commented on SPARK-15034: -- Seems your default file system is hdfs? Since we nee

[jira] [Closed] (SPARK-12893) RM redirects to incorrect URL in Spark HistoryServer for yarn-cluster mode

2016-05-25 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao closed SPARK-12893. --- Resolution: Not A Problem > RM redirects to incorrect URL in Spark HistoryServer for yarn-cluster mod

[jira] [Commented] (SPARK-15034) Use the value of spark.sql.warehouse.dir as the warehouse location instead of using hive.metastore.warehouse.dir

2016-05-25 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301508#comment-15301508 ] koert kuipers commented on SPARK-15034: --- this just hit me today. i build spark 2.0.

[jira] [Created] (SPARK-15546) HiveContext : Connecting to MySQL metastore db - Always creates a DERBY database in first attempt

2016-05-25 Thread Jais Sebastian (JIRA)
Jais Sebastian created SPARK-15546: -- Summary: HiveContext : Connecting to MySQL metastore db - Always creates a DERBY database in first attempt Key: SPARK-15546 URL: https://issues.apache.org/jira/browse/SPARK-1

[jira] [Updated] (SPARK-15546) HiveContext : Connecting to MySQL metastore db - Always creates a DERBY database in first attempt

2016-05-25 Thread Jais Sebastian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jais Sebastian updated SPARK-15546: --- Attachment: log.txt hive-site.xml > HiveContext : Connecting to MySQL metasto

[jira] [Resolved] (SPARK-15439) Failed to run unit test in SparkR

2016-05-25 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-15439. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request

[jira] [Updated] (SPARK-15439) Failed to run unit test in SparkR

2016-05-25 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-15439: -- Assignee: Miao Wang > Failed to run unit test in SparkR > -

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301468#comment-15301468 ] Yi Zhou commented on SPARK-15345: - I'm confused that it worked well with Apache Spark 1.6

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-25 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301462#comment-15301462 ] Jeff Zhang commented on SPARK-15345: This seems hadoop distributor specific, you may

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301456#comment-15301456 ] Yi Zhou commented on SPARK-15345: - I issued 'show databases;' , ' use XXX' and 'show tabl

[jira] [Commented] (SPARK-15541) SparkContext.stop throws error

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301457#comment-15301457 ] Sean Owen commented on SPARK-15541: --- Yes, this is a subtle Java version issue. It means

[jira] [Updated] (SPARK-12504) JDBC data source credentials are not masked in the data frame explain output.

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12504: -- Assignee: Suresh Thalamati (was: Apache Spark) > JDBC data source credentials are not masked in the da

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-25 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301437#comment-15301437 ] Jeff Zhang commented on SPARK-15345: bq. but it can't work after deploying the Apache

[jira] [Updated] (SPARK-15545) R remove non-exported unused methods, like jsonRDD

2016-05-25 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-15545: - Description: Need to review what should be removed. one reason to not remove this right away is b

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301427#comment-15301427 ] Yi Zhou commented on SPARK-15345: - Hi [~jeffzhang] In my test environment, i installed CD

[jira] [Created] (SPARK-15545) R remove non-exported unused methods, like jsonRDD

2016-05-25 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-15545: Summary: R remove non-exported unused methods, like jsonRDD Key: SPARK-15545 URL: https://issues.apache.org/jira/browse/SPARK-15545 Project: Spark Issue Type

[jira] [Commented] (SPARK-12504) JDBC data source credentials are not masked in the data frame explain output.

2016-05-25 Thread Suresh Thalamati (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301392#comment-15301392 ] Suresh Thalamati commented on SPARK-12504: -- [~srowen] I worked on resolving th

[jira] [Commented] (SPARK-15541) SparkContext.stop throws error

2016-05-25 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301352#comment-15301352 ] Saisai Shao commented on SPARK-15541: - This method which returns {{KeySetView}} only

[jira] [Commented] (SPARK-13955) Spark in yarn mode fails

2016-05-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301342#comment-15301342 ] Yi Zhou commented on SPARK-13955: - Thanks a lot [~jerryshao]! > Spark in yarn mode fails

[jira] [Assigned] (SPARK-15129) Clarify conventions for calling Spark and MLlib from R

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15129: Assignee: (was: Apache Spark) > Clarify conventions for calling Spark and MLlib from R

[jira] [Commented] (SPARK-15129) Clarify conventions for calling Spark and MLlib from R

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301322#comment-15301322 ] Apache Spark commented on SPARK-15129: -- User 'GayathriMurali' has created a pull req

[jira] [Assigned] (SPARK-15129) Clarify conventions for calling Spark and MLlib from R

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15129: Assignee: Apache Spark > Clarify conventions for calling Spark and MLlib from R >

[jira] [Resolved] (SPARK-15533) Deprecate Dataset.explode

2016-05-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15533. - Resolution: Fixed Assignee: Sameer Agarwal Fix Version/s: 2.0.0 > Deprecate Datas

[jira] [Reopened] (SPARK-15496) Spill metrics not updated when off-heap memory is enabled

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-15496: --- > Spill metrics not updated when off-heap memory is enabled > ---

[jira] [Updated] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15285: -- Assignee: Kazuaki Ishizaki > Generated SpecificSafeProjection.apply method grows beyond 64 KB > ---

[jira] [Resolved] (SPARK-15496) Spill metrics not updated when off-heap memory is enabled

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15496. --- Resolution: Cannot Reproduce Fix Version/s: (was: 2.0.0) > Spill metrics not updated when

[jira] [Updated] (SPARK-15315) CSV datasource writes garbage for complex types instead of rasing error.

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15315: -- Assignee: Suresh Thalamati > CSV datasource writes garbage for complex types instead of rasing error.

[jira] [Updated] (SPARK-14292) TPC-DS Query 23 Fails with compile error

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-14292: -- Assignee: Herman van Hovell > TPC-DS Query 23 Fails with compile error > --

[jira] [Commented] (SPARK-14292) TPC-DS Query 23 Fails with compile error

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301308#comment-15301308 ] Sean Owen commented on SPARK-14292: --- We should really link this to the JIRA/change that

[jira] [Updated] (SPARK-13820) TPC-DS Query 10 fails to compile

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13820: -- Assignee: Davies Liu > TPC-DS Query 10 fails to compile > > >

[jira] [Updated] (SPARK-12071) Programming guide should explain NULL in JVM translate to NA in R

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12071: -- Assignee: Krishna Kalyan > Programming guide should explain NULL in JVM translate to NA in R >

[jira] [Updated] (SPARK-10043) Add window functions into SparkR

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10043: -- Assignee: Sun Rui > Add window functions into SparkR > > >

[jira] [Commented] (SPARK-15526) Shade JPMML

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301298#comment-15301298 ] Sean Owen commented on SPARK-15526: --- I've also shaded JPMML in an app in order to work

[jira] [Updated] (SPARK-6009) IllegalArgumentException thrown by TimSort when SQL ORDER BY RAND ()

2016-05-25 Thread Sameer Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sameer Agarwal updated SPARK-6009: -- Fix Version/s: 1.5.0 > IllegalArgumentException thrown by TimSort when SQL ORDER BY RAND () > --

[jira] [Resolved] (SPARK-15527) Duplicate column names with different case after join of DataFrames

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15527. --- Resolution: Not A Problem > Duplicate column names with different case after join of DataFrames > ---

[jira] [Reopened] (SPARK-15527) Duplicate column names with different case after join of DataFrames

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-15527: --- Oops, I think maybe we use "Not A Problem" in this case, but the meaning is clear here > Duplicate colu

[jira] [Resolved] (SPARK-15527) Duplicate column names with different case after join of DataFrames

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15527. --- Resolution: Fixed OK, did you mean this is resolved? normally we wouldn't make a JIRA for it but I u

[jira] [Resolved] (SPARK-15332) OutOfMemory in TimSort

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15332. --- Resolution: Duplicate > OutOfMemory in TimSort > --- > > Key: SP

[jira] [Commented] (SPARK-12550) sbt-launch-lib.bash: line 72: 2404 Killed "$@"

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301273#comment-15301273 ] Sean Owen commented on SPARK-12550: --- I can't reproduce this. How are you building; what

[jira] [Resolved] (SPARK-15524) Strange issue starting spark-shell: screen block

2016-05-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15524. --- Resolution: Not A Problem This is specific to your environment. Your config is specifying a wrong/hos

[jira] [Updated] (SPARK-15544) Bouncing Zookeeper node causes Active spark master to exit

2016-05-25 Thread Steven Lowenthal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steven Lowenthal updated SPARK-15544: - Description: Shutting Down a single zookeeper node caused spark master to exit. The mast

[jira] [Updated] (SPARK-15544) Bouncing Zookeeper node causes Active spark master to exit

2016-05-25 Thread Steven Lowenthal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steven Lowenthal updated SPARK-15544: - Description: Shutting Down a single zookeeper node caused spark master to exit. The mast

[jira] [Created] (SPARK-15544) Bouncing Zookeeper node causes Active spark master to exit

2016-05-25 Thread Steven Lowenthal (JIRA)
Steven Lowenthal created SPARK-15544: Summary: Bouncing Zookeeper node causes Active spark master to exit Key: SPARK-15544 URL: https://issues.apache.org/jira/browse/SPARK-15544 Project: Spark

[jira] [Commented] (SPARK-15539) DROP TABLE should throw exceptions, not logError

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301233#comment-15301233 ] Apache Spark commented on SPARK-15539: -- User 'andrewor14' has created a pull request

[jira] [Assigned] (SPARK-15536) Disallow TRUNCATE TABLE with external tables and views

2016-05-25 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reassigned SPARK-15536: - Assignee: Andrew Or (was: Suresh Thalamati) > Disallow TRUNCATE TABLE with external tables and

[jira] [Updated] (SPARK-15538) Truncate table does not work on data source table

2016-05-25 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15538: -- Affects Version/s: 2.0.0 Target Version/s: 2.0.0 > Truncate table does not work on data source tab

[jira] [Updated] (SPARK-15536) Disallow TRUNCATE TABLE with external tables and views

2016-05-25 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15536: -- Summary: Disallow TRUNCATE TABLE with external tables and views (was: Disallow TRUNCATE TABLE with ext

[jira] [Updated] (SPARK-15538) Truncate table does not work on data source table

2016-05-25 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15538: -- Description: Truncate table does not seem to work on data source tables. Repro: {code} val df = Seq((1

[jira] [Updated] (SPARK-15538) Truncate table does not work on data source table

2016-05-25 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15538: -- Description: Truncate table does not seems to work on data source table. It returns success without a

[jira] [Commented] (SPARK-15457) Eliminate MLlib 2.0 build warnings from deprecations

2016-05-25 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301223#comment-15301223 ] Joseph K. Bradley commented on SPARK-15457: --- I just sent a PR. It does not cle

[jira] [Commented] (SPARK-15538) Truncate table does not work on data source table

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301222#comment-15301222 ] Apache Spark commented on SPARK-15538: -- User 'andrewor14' has created a pull request

[jira] [Assigned] (SPARK-15457) Eliminate MLlib 2.0 build warnings from deprecations

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15457: Assignee: Joseph K. Bradley (was: Apache Spark) > Eliminate MLlib 2.0 build warnings from

[jira] [Updated] (SPARK-15538) Truncate table does not work on data source table

2016-05-25 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15538: -- Summary: Truncate table does not work on data source table (was: Truncate table does not work on data

[jira] [Commented] (SPARK-15457) Eliminate MLlib 2.0 build warnings from deprecations

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301218#comment-15301218 ] Apache Spark commented on SPARK-15457: -- User 'jkbradley' has created a pull request

[jira] [Assigned] (SPARK-15538) Truncate table does not work on data source table

2016-05-25 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reassigned SPARK-15538: - Assignee: Andrew Or (was: Suresh Thalamati) > Truncate table does not work on data source table

[jira] [Assigned] (SPARK-15457) Eliminate MLlib 2.0 build warnings from deprecations

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15457: Assignee: Apache Spark (was: Joseph K. Bradley) > Eliminate MLlib 2.0 build warnings from

[jira] [Updated] (SPARK-15457) Eliminate MLlib 2.0 build warnings from deprecations

2016-05-25 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15457: -- Description: Several classes and methods have been deprecated and are creating lots of

[jira] [Commented] (SPARK-15533) Deprecate Dataset.explode

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301215#comment-15301215 ] Apache Spark commented on SPARK-15533: -- User 'WeichenXu123' has created a pull reque

[jira] [Commented] (SPARK-13184) Support minPartitions parameter for JSON and CSV datasources as options

2016-05-25 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301208#comment-15301208 ] Takeshi Yamamuro commented on SPARK-13184: -- okay > Support minPartitions parame

[jira] [Commented] (SPARK-13184) Support minPartitions parameter for JSON and CSV datasources as options

2016-05-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301205#comment-15301205 ] Reynold Xin commented on SPARK-13184: - Please go for it. > Support minPartitions pa

[jira] [Commented] (SPARK-13184) Support minPartitions parameter for JSON and CSV datasources as options

2016-05-25 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301202#comment-15301202 ] Takeshi Yamamuro commented on SPARK-13184: -- [~rxin] Is this ticket worth a try?

[jira] [Commented] (SPARK-15533) Deprecate Dataset.explode

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301196#comment-15301196 ] Apache Spark commented on SPARK-15533: -- User 'sameeragarwal' has created a pull requ

[jira] [Assigned] (SPARK-15533) Deprecate Dataset.explode

2016-05-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15533: Assignee: (was: Apache Spark) > Deprecate Dataset.explode > -

  1   2   3   >