[jira] [Commented] (SPARK-13635) Enable LimitPushdown optimizer rule because we have whole-stage codegen for Limit

[ https://issues.apache.org/jira/browse/SPARK-13635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177445#comment-15177445 ] Liang-Chi Hsieh commented on SPARK-13635: - [~davies] Can you help update the Assignee field?

[jira] [Commented] (SPARK-13531) Some DataFrame joins stopped working with UnsupportedOperationException: No size estimation available for objects

[ https://issues.apache.org/jira/browse/SPARK-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177441#comment-15177441 ] Zuo Wang commented on SPARK-13531: -- Caused by the commit in

[jira] [Resolved] (SPARK-13635) Enable LimitPushdown optimizer rule because we have whole-stage codegen for Limit

[ https://issues.apache.org/jira/browse/SPARK-13635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13635. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11483

[jira] [Commented] (SPARK-13589) Flaky test: ParquetHadoopFsRelationSuite.test all data types - ByteType

[ https://issues.apache.org/jira/browse/SPARK-13589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177437#comment-15177437 ] Liang-Chi Hsieh commented on SPARK-13589: - [~lian cheng] I think this is already solved in

[jira] [Commented] (SPARK-12941) Spark-SQL JDBC Oracle dialect fails to map string datatypes to Oracle VARCHAR datatype

[ https://issues.apache.org/jira/browse/SPARK-12941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177436#comment-15177436 ] Apache Spark commented on SPARK-12941: -- User 'thomastechs' has created a pull request for this

[jira] [Comment Edited] (SPARK-13612) Multiplication of BigDecimal columns not working as expected

[ https://issues.apache.org/jira/browse/SPARK-13612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177428#comment-15177428 ] Liang-Chi Hsieh edited comment on SPARK-13612 at 3/3/16 7:35 AM: - Because

[jira] [Commented] (SPARK-13612) Multiplication of BigDecimal columns not working as expected

[ https://issues.apache.org/jira/browse/SPARK-13612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177428#comment-15177428 ] Liang-Chi Hsieh commented on SPARK-13612: - Because the internal type for BigDecimal would be

[jira] [Created] (SPARK-13643) Create SparkSession interface

Reynold Xin created SPARK-13643: --- Summary: Create SparkSession interface Key: SPARK-13643 URL: https://issues.apache.org/jira/browse/SPARK-13643 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-13600) Incorrect number of buckets in QuantileDiscretizer

[ https://issues.apache.org/jira/browse/SPARK-13600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177392#comment-15177392 ] Xusen Yin commented on SPARK-13600: --- Vote for the new method. > Incorrect number of buckets in

[jira] [Commented] (SPARK-13568) Create feature transformer to impute missing values

[ https://issues.apache.org/jira/browse/SPARK-13568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177368#comment-15177368 ] Nick Pentreath commented on SPARK-13568: Ok - the Imputer will need to compute column stats

[jira] [Commented] (SPARK-13446) Spark need to support reading data from Hive 2.0.0 metastore

[ https://issues.apache.org/jira/browse/SPARK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177356#comment-15177356 ] Adrian Wang commented on SPARK-13446: - That's not enough. We still need some code change. > Spark

[jira] [Commented] (SPARK-13311) prettyString of IN is not good

[ https://issues.apache.org/jira/browse/SPARK-13311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177357#comment-15177357 ] Xiao Li commented on SPARK-13311: - After the merge of https://github.com/apache/spark/pull/10757, I think

[jira] [Updated] (SPARK-13446) Spark need to support reading data from Hive 2.0.0 metastore

[ https://issues.apache.org/jira/browse/SPARK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13446: -- Issue Type: Improvement (was: Bug) Can't you build against the newer version of Hive? that much is

[jira] [Commented] (SPARK-13642) Inconsistent finishing state between driver and AM

[ https://issues.apache.org/jira/browse/SPARK-13642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177347#comment-15177347 ] Saisai Shao commented on SPARK-13642: - [~tgraves] [~vanzin], would you please comment on this, why

[jira] [Updated] (SPARK-13642) Inconsistent finishing state between driver and AM

[ https://issues.apache.org/jira/browse/SPARK-13642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-13642: Description: Currently when running Spark on Yarn with yarn cluster mode, the default application

[jira] [Created] (SPARK-13642) Inconsistent finishing state between driver and AM

Saisai Shao created SPARK-13642: --- Summary: Inconsistent finishing state between driver and AM Key: SPARK-13642 URL: https://issues.apache.org/jira/browse/SPARK-13642 Project: Spark Issue

[jira] [Resolved] (SPARK-13621) TestExecutor.scala needs to be moved to test package

[ https://issues.apache.org/jira/browse/SPARK-13621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-13621. - Resolution: Fixed Assignee: Devaraj K Fix Version/s: 2.0.0 > TestExecutor.scala

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177315#comment-15177315 ] Mark Grover commented on SPARK-12177: - One more thing as a potential con for Proposal 1: There are

[jira] [Resolved] (SPARK-13616) Let SQLBuilder convert logical plan without a Project on top of it

[ https://issues.apache.org/jira/browse/SPARK-13616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-13616. - Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 2.0.0 > Let

[jira] [Created] (SPARK-13641) getModelFeatures of ml.api.r.SparkRWrapper cannot (always) reveal the original column names

Xusen Yin created SPARK-13641: - Summary: getModelFeatures of ml.api.r.SparkRWrapper cannot (always) reveal the original column names Key: SPARK-13641 URL: https://issues.apache.org/jira/browse/SPARK-13641

[jira] [Updated] (SPARK-13446) Spark need to support reading data from Hive 2.0.0 metastore

[ https://issues.apache.org/jira/browse/SPARK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lifeng Wang updated SPARK-13446: Summary: Spark need to support reading data from Hive 2.0.0 metastore (was: Spark need to support

[jira] [Assigned] (SPARK-13640) Synchronize ScalaReflection.mirror method.

[ https://issues.apache.org/jira/browse/SPARK-13640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13640: Assignee: Apache Spark > Synchronize ScalaReflection.mirror method. >

[jira] [Commented] (SPARK-13640) Synchronize ScalaReflection.mirror method.

[ https://issues.apache.org/jira/browse/SPARK-13640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177301#comment-15177301 ] Apache Spark commented on SPARK-13640: -- User 'ueshin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13640) Synchronize ScalaReflection.mirror method.

[ https://issues.apache.org/jira/browse/SPARK-13640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13640: Assignee: (was: Apache Spark) > Synchronize ScalaReflection.mirror method. >

[jira] [Commented] (SPARK-13449) Naive Bayes wrapper in SparkR

[ https://issues.apache.org/jira/browse/SPARK-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177298#comment-15177298 ] Apache Spark commented on SPARK-13449: -- User 'yinxusen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13449) Naive Bayes wrapper in SparkR

[ https://issues.apache.org/jira/browse/SPARK-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13449: Assignee: Xusen Yin (was: Apache Spark) > Naive Bayes wrapper in SparkR >

[jira] [Assigned] (SPARK-13449) Naive Bayes wrapper in SparkR

[ https://issues.apache.org/jira/browse/SPARK-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13449: Assignee: Apache Spark (was: Xusen Yin) > Naive Bayes wrapper in SparkR >

[jira] [Commented] (SPARK-13631) getPreferredLocations race condition in spark 1.6.0?

[ https://issues.apache.org/jira/browse/SPARK-13631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177261#comment-15177261 ] Andy Sloane commented on SPARK-13631: - Did some digging with git bisect. It turns out to be directly

[jira] [Comment Edited] (SPARK-13568) Create feature transformer to impute missing values

[ https://issues.apache.org/jira/browse/SPARK-13568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172423#comment-15172423 ] yuhao yang edited comment on SPARK-13568 at 3/3/16 5:48 AM: Yes, I'm working

[jira] [Updated] (SPARK-13638) Support for saving with a quote mode

[ https://issues.apache.org/jira/browse/SPARK-13638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-13638: - Description: https://github.com/databricks/spark-csv/pull/254 tobithiel reported this. {quote}

[jira] [Updated] (SPARK-13638) Support for saving with a quote mode

[ https://issues.apache.org/jira/browse/SPARK-13638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-13638: - Description: https://github.com/databricks/spark-csv/pull/254 tobithiel reported this. {quote}

[jira] [Commented] (SPARK-13637) use more information to simplify the code in Expand builder

[ https://issues.apache.org/jira/browse/SPARK-13637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177256#comment-15177256 ] Apache Spark commented on SPARK-13637: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Updated] (SPARK-13638) Support for saving with a quote mode

[ https://issues.apache.org/jira/browse/SPARK-13638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-13638: - Description: https://github.com/databricks/spark-csv/pull/254 tobithiel reported this. {quote}

[jira] [Created] (SPARK-13640) Synchronize ScalaReflection.mirror method.

Takuya Ueshin created SPARK-13640: - Summary: Synchronize ScalaReflection.mirror method. Key: SPARK-13640 URL: https://issues.apache.org/jira/browse/SPARK-13640 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-13637) use more information to simplify the code in Expand builder

[ https://issues.apache.org/jira/browse/SPARK-13637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13637: Assignee: (was: Apache Spark) > use more information to simplify the code in Expand

[jira] [Assigned] (SPARK-13637) use more information to simplify the code in Expand builder

[ https://issues.apache.org/jira/browse/SPARK-13637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13637: Assignee: Apache Spark > use more information to simplify the code in Expand builder >

[jira] [Created] (SPARK-13639) Statistics.colStats(rdd).mean and variance should handle NaN in the input vectors

yuhao yang created SPARK-13639: -- Summary: Statistics.colStats(rdd).mean and variance should handle NaN in the input vectors Key: SPARK-13639 URL: https://issues.apache.org/jira/browse/SPARK-13639

[jira] [Created] (SPARK-13638) Support for saving with a quote mode

Hyukjin Kwon created SPARK-13638: Summary: Support for saving with a quote mode Key: SPARK-13638 URL: https://issues.apache.org/jira/browse/SPARK-13638 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-13637) use more information to simplify the code in Expand builder

Wenchen Fan created SPARK-13637: --- Summary: use more information to simplify the code in Expand builder Key: SPARK-13637 URL: https://issues.apache.org/jira/browse/SPARK-13637 Project: Spark

[jira] [Updated] (SPARK-13634) Assigning spark context to variable results in serialization error

[ https://issues.apache.org/jira/browse/SPARK-13634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13634: -- Priority: Minor (was: Major) I doubt it's a Spark problem; this is more a function of how Scala puts

[jira] [Commented] (SPARK-13602) o.a.s.deploy.worker.DriverRunner may leak the driver processes

[ https://issues.apache.org/jira/browse/SPARK-13602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177169#comment-15177169 ] Bryan Cutler commented on SPARK-13602: -- Great! Thanks :D > o.a.s.deploy.worker.DriverRunner may

[jira] [Updated] (SPARK-13600) Incorrect number of buckets in QuantileDiscretizer

[ https://issues.apache.org/jira/browse/SPARK-13600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13600: -- Assignee: Oliver Pierson > Incorrect number of buckets in QuantileDiscretizer >

[jira] [Commented] (SPARK-13636) Direct consume UnsafeRow in wholestage codegen plans

[ https://issues.apache.org/jira/browse/SPARK-13636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177146#comment-15177146 ] Apache Spark commented on SPARK-13636: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13636) Direct consume UnsafeRow in wholestage codegen plans

[ https://issues.apache.org/jira/browse/SPARK-13636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13636: Assignee: Apache Spark > Direct consume UnsafeRow in wholestage codegen plans >

[jira] [Resolved] (SPARK-13627) Fix simple deprecation warnings

[ https://issues.apache.org/jira/browse/SPARK-13627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-13627. - Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 2.0.0 > Fix simple

[jira] [Created] (SPARK-13636) Direct consume UnsafeRow in wholestage codegen plans

Liang-Chi Hsieh created SPARK-13636: --- Summary: Direct consume UnsafeRow in wholestage codegen plans Key: SPARK-13636 URL: https://issues.apache.org/jira/browse/SPARK-13636 Project: Spark

[jira] [Resolved] (SPARK-13617) remove unnecessary GroupingAnalytics trait

[ https://issues.apache.org/jira/browse/SPARK-13617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-13617. - Resolution: Fixed Assignee: Wenchen Fan Fix Version/s: 2.0.0 > remove

[jira] [Assigned] (SPARK-13635) Enable LimitPushdown optimizer rule because we have whole-stage codegen for Limit

[ https://issues.apache.org/jira/browse/SPARK-13635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13635: Assignee: (was: Apache Spark) > Enable LimitPushdown optimizer rule because we have

[jira] [Assigned] (SPARK-13635) Enable LimitPushdown optimizer rule because we have whole-stage codegen for Limit

[ https://issues.apache.org/jira/browse/SPARK-13635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13635: Assignee: Apache Spark > Enable LimitPushdown optimizer rule because we have whole-stage

[jira] [Commented] (SPARK-13635) Enable LimitPushdown optimizer rule because we have whole-stage codegen for Limit

[ https://issues.apache.org/jira/browse/SPARK-13635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177108#comment-15177108 ] Apache Spark commented on SPARK-13635: -- User 'viirya' has created a pull request for this issue:

[jira] [Created] (SPARK-13635) Enable LimitPushdown optimizer rule because we have whole-stage codegen for Limit

Liang-Chi Hsieh created SPARK-13635: --- Summary: Enable LimitPushdown optimizer rule because we have whole-stage codegen for Limit Key: SPARK-13635 URL: https://issues.apache.org/jira/browse/SPARK-13635

[jira] [Updated] (SPARK-13593) improve the `toDF()` method to accept data type string and verify the data

[ https://issues.apache.org/jira/browse/SPARK-13593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-13593: Summary: improve the `toDF()` method to accept data type string and verify the data (was: add a

[jira] [Updated] (SPARK-13634) Assigning spark context to variable results in serialization error

[ https://issues.apache.org/jira/browse/SPARK-13634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rahul Palamuttam updated SPARK-13634: - Description: The following lines of code cause a task serialization error when executed

[jira] [Commented] (SPARK-13634) Assigning spark context to variable results in serialization error

[ https://issues.apache.org/jira/browse/SPARK-13634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177093#comment-15177093 ] Rahul Palamuttam commented on SPARK-13634: -- [~chrismattmann] > Assigning spark context to

[jira] [Created] (SPARK-13634) Assigning spark context to variable results in serialization error

Rahul Palamuttam created SPARK-13634: Summary: Assigning spark context to variable results in serialization error Key: SPARK-13634 URL: https://issues.apache.org/jira/browse/SPARK-13634 Project:

[jira] [Commented] (SPARK-13632) Create new o.a.s.sql.execution.commands package

[ https://issues.apache.org/jira/browse/SPARK-13632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177082#comment-15177082 ] Apache Spark commented on SPARK-13632: -- User 'andrewor14' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13632) Create new o.a.s.sql.execution.commands package

[ https://issues.apache.org/jira/browse/SPARK-13632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13632: Assignee: Apache Spark (was: Andrew Or) > Create new o.a.s.sql.execution.commands

[jira] [Assigned] (SPARK-13632) Create new o.a.s.sql.execution.commands package

[ https://issues.apache.org/jira/browse/SPARK-13632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13632: Assignee: Andrew Or (was: Apache Spark) > Create new o.a.s.sql.execution.commands

[jira] [Updated] (SPARK-13633) Move parser classes to o.a.s.sql.catalyst.parser package

[ https://issues.apache.org/jira/browse/SPARK-13633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13633: -- Summary: Move parser classes to o.a.s.sql.catalyst.parser package (was: Create new

[jira] [Created] (SPARK-13633) Create new o.a.s.sql.catalyst.parser package

Andrew Or created SPARK-13633: - Summary: Create new o.a.s.sql.catalyst.parser package Key: SPARK-13633 URL: https://issues.apache.org/jira/browse/SPARK-13633 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-13632) Create new o.a.s.sql.execution.commands package

Andrew Or created SPARK-13632: - Summary: Create new o.a.s.sql.execution.commands package Key: SPARK-13632 URL: https://issues.apache.org/jira/browse/SPARK-13632 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-13614) show() trigger memory leak,why?

[ https://issues.apache.org/jira/browse/SPARK-13614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15175516#comment-15175516 ] chillon_m edited comment on SPARK-13614 at 3/3/16 2:16 AM: --- @[~srowen] the same

[jira] [Comment Edited] (SPARK-13614) show() trigger memory leak,why?

[ https://issues.apache.org/jira/browse/SPARK-13614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15175516#comment-15175516 ] chillon_m edited comment on SPARK-13614 at 3/3/16 2:14 AM: --- @[~srowen] the same

[jira] [Comment Edited] (SPARK-13614) show() trigger memory leak,why?

[ https://issues.apache.org/jira/browse/SPARK-13614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15175516#comment-15175516 ] chillon_m edited comment on SPARK-13614 at 3/3/16 2:14 AM: --- [~srowen] the same

[jira] [Issue Comment Deleted] (SPARK-13614) show() trigger memory leak,why?

[ https://issues.apache.org/jira/browse/SPARK-13614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chillon_m updated SPARK-13614: -- Comment: was deleted (was: the same size of dataset,collect don't trigger memory leak(first

[jira] [Updated] (SPARK-13485) (Dataset-oriented) API evolution in Spark 2.0

[ https://issues.apache.org/jira/browse/SPARK-13485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-13485: Summary: (Dataset-oriented) API evolution in Spark 2.0 (was: Dataset-oriented API foundation in

[jira] [Updated] (SPARK-13485) (Dataset-oriented) API evolution in Spark 2.0

[ https://issues.apache.org/jira/browse/SPARK-13485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-13485: Description: As part of Spark 2.0, we want to create a stable API foundation for Dataset to

[jira] [Updated] (SPARK-13485) Dataset-oriented API foundation in Spark 2.0

[ https://issues.apache.org/jira/browse/SPARK-13485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-13485: Attachment: API Evolution in Spark 2.0.pdf > Dataset-oriented API foundation in Spark 2.0 >

[jira] [Updated] (SPARK-13583) Remove unused imports and add checkstyle rule

[ https://issues.apache.org/jira/browse/SPARK-13583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-13583: -- Summary: Remove unused imports and add checkstyle rule (was: Support `UnusedImports` Java

[jira] [Commented] (SPARK-13602) o.a.s.deploy.worker.DriverRunner may leak the driver processes

[ https://issues.apache.org/jira/browse/SPARK-13602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176844#comment-15176844 ] Shixiong Zhu commented on SPARK-13602: -- Sure. Go ahead. > o.a.s.deploy.worker.DriverRunner may leak

[jira] [Updated] (SPARK-13627) Fix simple deprecation warnings

[ https://issues.apache.org/jira/browse/SPARK-13627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-13627: -- Component/s: (was: PySpark) > Fix simple deprecation warnings >

[jira] [Commented] (SPARK-13630) Add optimizer rule to collapse sorts

[ https://issues.apache.org/jira/browse/SPARK-13630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176810#comment-15176810 ] Apache Spark commented on SPARK-13630: -- User 'skambha' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13630) Add optimizer rule to collapse sorts

[ https://issues.apache.org/jira/browse/SPARK-13630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13630: Assignee: (was: Apache Spark) > Add optimizer rule to collapse sorts >

[jira] [Commented] (SPARK-13630) Add optimizer rule to collapse sorts

[ https://issues.apache.org/jira/browse/SPARK-13630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176811#comment-15176811 ] Sunitha Kambhampati commented on SPARK-13630: - Here is the pull request with changes:

[jira] [Assigned] (SPARK-13630) Add optimizer rule to collapse sorts

[ https://issues.apache.org/jira/browse/SPARK-13630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13630: Assignee: Apache Spark > Add optimizer rule to collapse sorts >

[jira] [Created] (SPARK-13631) getPreferredLocations race condition in spark 1.6.0?

Andy Sloane created SPARK-13631: --- Summary: getPreferredLocations race condition in spark 1.6.0? Key: SPARK-13631 URL: https://issues.apache.org/jira/browse/SPARK-13631 Project: Spark Issue

[jira] [Commented] (SPARK-13627) Fix simple deprecation warnings

[ https://issues.apache.org/jira/browse/SPARK-13627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176802#comment-15176802 ] Apache Spark commented on SPARK-13627: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-13627) Fix simple deprecation warnings

[ https://issues.apache.org/jira/browse/SPARK-13627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13627: Assignee: (was: Apache Spark) > Fix simple deprecation warnings >

[jira] [Assigned] (SPARK-13627) Fix simple deprecation warnings

[ https://issues.apache.org/jira/browse/SPARK-13627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13627: Assignee: Apache Spark > Fix simple deprecation warnings >

[jira] [Commented] (SPARK-13602) o.a.s.deploy.worker.DriverRunner may leak the driver processes

[ https://issues.apache.org/jira/browse/SPARK-13602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176787#comment-15176787 ] Bryan Cutler commented on SPARK-13602: -- Hi [~zsxwing], mind if I work on this one? >

[jira] [Created] (SPARK-13630) Add optimizer rule to collapse sorts

Sunitha Kambhampati created SPARK-13630: --- Summary: Add optimizer rule to collapse sorts Key: SPARK-13630 URL: https://issues.apache.org/jira/browse/SPARK-13630 Project: Spark Issue

[jira] [Updated] (SPARK-13627) Fix simple deprecation warnings

[ https://issues.apache.org/jira/browse/SPARK-13627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-13627: -- Description: This issue aims to fix the following deprecation warnings. *

[jira] [Created] (SPARK-13629) Add binary toggle Param to CountVectorizer

Joseph K. Bradley created SPARK-13629: - Summary: Add binary toggle Param to CountVectorizer Key: SPARK-13629 URL: https://issues.apache.org/jira/browse/SPARK-13629 Project: Spark Issue

[jira] [Created] (SPARK-13628) Temporary intermediate output file should be renamed before copying to destination filesystem

Chen He created SPARK-13628: --- Summary: Temporary intermediate output file should be renamed before copying to destination filesystem Key: SPARK-13628 URL: https://issues.apache.org/jira/browse/SPARK-13628

[jira] [Commented] (SPARK-13465) Add a task failure listener to TaskContext

[ https://issues.apache.org/jira/browse/SPARK-13465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176750#comment-15176750 ] Apache Spark commented on SPARK-13465: -- User 'davies' has created a pull request for this issue:

[jira] [Commented] (SPARK-13161) Extend MLlib LDA to include options for Author Topic Modeling

[ https://issues.apache.org/jira/browse/SPARK-13161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176747#comment-15176747 ] Joseph K. Bradley commented on SPARK-13161: --- There are many generalizations of LDA, so it would

[jira] [Updated] (SPARK-13161) Extend MLlib LDA to include options for Author Topic Modeling

[ https://issues.apache.org/jira/browse/SPARK-13161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-13161: -- Priority: Minor (was: Major) > Extend MLlib LDA to include options for Author Topic

[jira] [Created] (SPARK-13627) Fix simple deprecation warnings

Dongjoon Hyun created SPARK-13627: - Summary: Fix simple deprecation warnings Key: SPARK-13627 URL: https://issues.apache.org/jira/browse/SPARK-13627 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-12925) Improve HiveInspectors.unwrap for StringObjectInspector.getPrimitiveWritableObject

[ https://issues.apache.org/jira/browse/SPARK-12925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176734#comment-15176734 ] Rajesh Balamohan commented on SPARK-12925: -- Earlier fix had a problem when Text was reused.

[jira] [Commented] (SPARK-12925) Improve HiveInspectors.unwrap for StringObjectInspector.getPrimitiveWritableObject

[ https://issues.apache.org/jira/browse/SPARK-12925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176729#comment-15176729 ] Apache Spark commented on SPARK-12925: -- User 'rajeshbalamohan' has created a pull request for this

[jira] [Created] (SPARK-13626) SparkConf deprecation log messages are printed multiple times

Marcelo Vanzin created SPARK-13626: -- Summary: SparkConf deprecation log messages are printed multiple times Key: SPARK-13626 URL: https://issues.apache.org/jira/browse/SPARK-13626 Project: Spark

[jira] [Assigned] (SPARK-13625) PySpark-ML method to get list of params for an obj should not check property attr

[ https://issues.apache.org/jira/browse/SPARK-13625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13625: Assignee: Apache Spark > PySpark-ML method to get list of params for an obj should not

[jira] [Assigned] (SPARK-13625) PySpark-ML method to get list of params for an obj should not check property attr

[ https://issues.apache.org/jira/browse/SPARK-13625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13625: Assignee: (was: Apache Spark) > PySpark-ML method to get list of params for an obj

[jira] [Commented] (SPARK-13625) PySpark-ML method to get list of params for an obj should not check property attr

[ https://issues.apache.org/jira/browse/SPARK-13625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176715#comment-15176715 ] Apache Spark commented on SPARK-13625: -- User 'BryanCutler' has created a pull request for this

[jira] [Resolved] (SPARK-13528) Make the short names of compression codecs consistent in spark

[ https://issues.apache.org/jira/browse/SPARK-13528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-13528. - Resolution: Fixed Assignee: Takeshi Yamamuro Fix Version/s: 2.0.0 > Make the

[jira] [Updated] (SPARK-13594) remove typed operations (map, flatMap, mapPartitions) from Python DataFrame

[ https://issues.apache.org/jira/browse/SPARK-13594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-13594: Description: Once we implement Dataset-equivalent API in Python, we'd need to change the return

[jira] [Updated] (SPARK-13594) remove typed operations (map, flatMap, mapPartitions) from Python DataFrame

[ https://issues.apache.org/jira/browse/SPARK-13594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-13594: Issue Type: Sub-task (was: Improvement) Parent: SPARK-11806 > remove typed operations

[jira] [Resolved] (SPARK-13594) remove typed operations (map, flatMap, mapPartitions) from Python DataFrame

[ https://issues.apache.org/jira/browse/SPARK-13594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-13594. - Resolution: Fixed Assignee: Wenchen Fan Fix Version/s: 2.0.0 > remove typed

[jira] [Updated] (SPARK-13594) remove typed operations(e.g. map, flatMap) from Python DataFrame

[ https://issues.apache.org/jira/browse/SPARK-13594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-13594: Summary: remove typed operations(e.g. map, flatMap) from Python DataFrame (was: remove typed

[jira] [Updated] (SPARK-13594) remove typed operations (map, flatMap, mapPartitions) from Python DataFrame

[ https://issues.apache.org/jira/browse/SPARK-13594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-13594: Summary: remove typed operations (map, flatMap, mapPartitions) from Python DataFrame (was:

  1   2   3   >