[jira] [Commented] (SPARK-7606) Document all PySpark SQL/DataFrame public methods with @since tag

[ https://issues.apache.org/jira/browse/SPARK-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547079#comment-14547079 ] Davies Liu commented on SPARK-7606: --- +1 for `versionadded` Document all PySpark

[jira] [Created] (SPARK-7686) Runnable DescribeCommand is assigned wrong physical plan output attributes in SparkStrategies

Josh Rosen created SPARK-7686: - Summary: Runnable DescribeCommand is assigned wrong physical plan output attributes in SparkStrategies Key: SPARK-7686 URL: https://issues.apache.org/jira/browse/SPARK-7686

[jira] [Assigned] (SPARK-7686) Runnable DescribeCommand is assigned wrong physical plan output attributes in SparkStrategies

[ https://issues.apache.org/jira/browse/SPARK-7686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7686: --- Assignee: (was: Apache Spark) Runnable DescribeCommand is assigned wrong physical plan

[jira] [Commented] (SPARK-7686) Runnable DescribeCommand is assigned wrong physical plan output attributes in SparkStrategies

[ https://issues.apache.org/jira/browse/SPARK-7686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547085#comment-14547085 ] Apache Spark commented on SPARK-7686: - User 'JoshRosen' has created a pull request for

[jira] [Assigned] (SPARK-7686) Runnable DescribeCommand is assigned wrong physical plan output attributes in SparkStrategies

[ https://issues.apache.org/jira/browse/SPARK-7686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7686: --- Assignee: Apache Spark Runnable DescribeCommand is assigned wrong physical plan output

[jira] [Updated] (SPARK-7686) Runnable DescribeCommand is assigned wrong physical plan output attributes in SparkStrategies

[ https://issues.apache.org/jira/browse/SPARK-7686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7686: --- Priority: Critical (was: Minor) Runnable DescribeCommand is assigned wrong physical plan output

[jira] [Updated] (SPARK-7686) Runnable DescribeCommand is assigned wrong physical plan output attributes in SparkStrategies

[ https://issues.apache.org/jira/browse/SPARK-7686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7686: --- Target Version/s: 1.3.2, 1.4.0 Runnable DescribeCommand is assigned wrong physical plan output

[jira] [Updated] (SPARK-7686) Runnable DescribeCommand is assigned wrong physical plan output attributes in SparkStrategies

[ https://issues.apache.org/jira/browse/SPARK-7686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7686: --- Assignee: Josh Rosen Runnable DescribeCommand is assigned wrong physical plan output attributes in

[jira] [Assigned] (SPARK-6785) DateUtils can not handle date before 1970/01/01 correctly

[ https://issues.apache.org/jira/browse/SPARK-6785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-6785: - Assignee: Davies Liu DateUtils can not handle date before 1970/01/01 correctly

[jira] [Updated] (SPARK-6785) DateUtils can not handle date before 1970/01/01 correctly

[ https://issues.apache.org/jira/browse/SPARK-6785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-6785: -- Assignee: (was: Christian Tzolov) DateUtils can not handle date before 1970/01/01 correctly

[jira] [Updated] (SPARK-6785) DateUtils can not handle date before 1970/01/01 correctly

[ https://issues.apache.org/jira/browse/SPARK-6785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-6785: -- Assignee: Christian Tzolov (was: Davies Liu) DateUtils can not handle date before 1970/01/01

[jira] [Created] (SPARK-7687) DataFrame.describe should cast all aggregates to doubles

Josh Rosen created SPARK-7687: - Summary: DataFrame.describe should cast all aggregates to doubles Key: SPARK-7687 URL: https://issues.apache.org/jira/browse/SPARK-7687 Project: Spark Issue Type:

[jira] [Updated] (SPARK-7687) DataFrame.describe() should cast all aggregates to doubles

[ https://issues.apache.org/jira/browse/SPARK-7687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-7687: -- Summary: DataFrame.describe() should cast all aggregates to doubles (was: DataFrame.describe should

[jira] [Commented] (SPARK-7687) DataFrame.describe() should cast all aggregates to doubles

[ https://issues.apache.org/jira/browse/SPARK-7687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547093#comment-14547093 ] Apache Spark commented on SPARK-7687: - User 'JoshRosen' has created a pull request for

[jira] [Assigned] (SPARK-7687) DataFrame.describe() should cast all aggregates to doubles

[ https://issues.apache.org/jira/browse/SPARK-7687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7687: --- Assignee: Josh Rosen (was: Apache Spark) DataFrame.describe() should cast all aggregates

[jira] [Assigned] (SPARK-7687) DataFrame.describe() should cast all aggregates to doubles

[ https://issues.apache.org/jira/browse/SPARK-7687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7687: --- Assignee: Apache Spark (was: Josh Rosen) DataFrame.describe() should cast all aggregates

[jira] [Commented] (SPARK-7110) when use saveAsNewAPIHadoopFile, sometimes it throws Delegation Token can be issued only with kerberos or web authentication

[ https://issues.apache.org/jira/browse/SPARK-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547098#comment-14547098 ] gu-chi commented on SPARK-7110: --- sorry, was busy these days Actually, I tried to reproduce

[jira] [Commented] (SPARK-7110) when use saveAsNewAPIHadoopFile, sometimes it throws Delegation Token can be issued only with kerberos or web authentication

[ https://issues.apache.org/jira/browse/SPARK-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547099#comment-14547099 ] gu-chi commented on SPARK-7110: --- sorry, was busy these days Actually, I tried to reproduce

[jira] [Issue Comment Deleted] (SPARK-7110) when use saveAsNewAPIHadoopFile, sometimes it throws Delegation Token can be issued only with kerberos or web authentication

[ https://issues.apache.org/jira/browse/SPARK-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gu-chi updated SPARK-7110: -- Comment: was deleted (was: sorry, was busy these days Actually, I tried to reproduce this issue for long time,

[jira] [Updated] (SPARK-7669) Builds against Hadoop 2.6+ get inconsistent curator dependencies

[ https://issues.apache.org/jira/browse/SPARK-7669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7669: - Assignee: Steve Loughran Builds against Hadoop 2.6+ get inconsistent curator dependencies

[jira] [Resolved] (SPARK-7669) Builds against Hadoop 2.6+ get inconsistent curator dependencies

[ https://issues.apache.org/jira/browse/SPARK-7669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-7669. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 6191

[jira] [Updated] (SPARK-6654) Update Kinesis Streaming impls (both KCL-based and Direct) to use latest aws-java-sdk and kinesis-client-library

[ https://issues.apache.org/jira/browse/SPARK-6654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Fregly updated SPARK-6654: Target Version/s: 1.4.0 (was: 1.5.0) Update Kinesis Streaming impls (both KCL-based and Direct)

[jira] [Commented] (SPARK-6681) JAVA_HOME error with upgrade to Spark 1.3.0

[ https://issues.apache.org/jira/browse/SPARK-6681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547182#comment-14547182 ] Olivier Armand commented on SPARK-6681: --- I resolved the issue on my side by making

[jira] [Commented] (SPARK-6416) RDD.fold() requires the operator to be commutative

[ https://issues.apache.org/jira/browse/SPARK-6416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547199#comment-14547199 ] Josh Rosen commented on SPARK-6416: --- Hey Sean, I don't think that this will be easy to

[jira] [Updated] (SPARK-7687) DataFrame.describe() should cast all aggregates to doubles

[ https://issues.apache.org/jira/browse/SPARK-7687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-7687: -- Target Version/s: 1.3.2, 1.4.0 DataFrame.describe() should cast all aggregates to doubles

[jira] [Updated] (SPARK-7687) DataFrame.describe() should cast all aggregates to doubles

[ https://issues.apache.org/jira/browse/SPARK-7687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-7687: -- Priority: Critical (was: Major) DataFrame.describe() should cast all aggregates to doubles

[jira] [Updated] (SPARK-6416) RDD.fold() requires the operator to be commutative

[ https://issues.apache.org/jira/browse/SPARK-6416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6416: - Target Version/s: 2+ RDD.fold() requires the operator to be commutative

[jira] [Created] (SPARK-7688) PySpark + ipython throws port out of range exception

Xiangrui Meng created SPARK-7688: Summary: PySpark + ipython throws port out of range exception Key: SPARK-7688 URL: https://issues.apache.org/jira/browse/SPARK-7688 Project: Spark Issue

[jira] [Updated] (SPARK-7688) PySpark + ipython throws port out of range exception

[ https://issues.apache.org/jira/browse/SPARK-7688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-7688: - Assignee: Davies Liu PySpark + ipython throws port out of range exception

[jira] [Updated] (SPARK-7689) Deprecate spark.cleaner.ttl

[ https://issues.apache.org/jira/browse/SPARK-7689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-7689: -- Component/s: Spark Core Deprecate spark.cleaner.ttl ---

[jira] [Created] (SPARK-7689) Deprecate spark.cleaner.ttl

Josh Rosen created SPARK-7689: - Summary: Deprecate spark.cleaner.ttl Key: SPARK-7689 URL: https://issues.apache.org/jira/browse/SPARK-7689 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-6654) Update Kinesis Streaming impls (both KCL-based and Direct) to use latest aws-java-sdk and kinesis-client-library

[ https://issues.apache.org/jira/browse/SPARK-6654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Fregly resolved SPARK-6654. - Resolution: Duplicate Fix Version/s: 1.4.0 duplicate of SPARK-7679 Update Kinesis

[jira] [Resolved] (SPARK-7660) Snappy-java buffer-sharing bug leads to data corruption / test failures

[ https://issues.apache.org/jira/browse/SPARK-7660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-7660. --- Resolution: Fixed Fix Version/s: 1.4.0 1.3.2 1.2.3 Fixed

[jira] [Commented] (SPARK-7689) Deprecate spark.cleaner.ttl

[ https://issues.apache.org/jira/browse/SPARK-7689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547264#comment-14547264 ] Apache Spark commented on SPARK-7689: - User 'JoshRosen' has created a pull request for

[jira] [Assigned] (SPARK-7689) Deprecate spark.cleaner.ttl

[ https://issues.apache.org/jira/browse/SPARK-7689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7689: --- Assignee: (was: Apache Spark) Deprecate spark.cleaner.ttl ---

[jira] [Assigned] (SPARK-7689) Deprecate spark.cleaner.ttl

[ https://issues.apache.org/jira/browse/SPARK-7689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7689: --- Assignee: Apache Spark Deprecate spark.cleaner.ttl ---

[jira] [Updated] (SPARK-7696) Aggregate function's result should be nullable only if the input expression is nullable

[ https://issues.apache.org/jira/browse/SPARK-7696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haopu Wang updated SPARK-7696: -- Description: In SparkSQL, the aggregate function's result currently is always nullable. It will make

[jira] [Commented] (SPARK-7063) Update lz4 for Java 7 to avoid: when lz4 compression is used, it causes core dump

[ https://issues.apache.org/jira/browse/SPARK-7063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547479#comment-14547479 ] Apache Spark commented on SPARK-7063: - User 'JihongMA' has created a pull request for

[jira] [Commented] (SPARK-6707) Mesos Scheduler should allow the user to specify constraints based on slave attributes

[ https://issues.apache.org/jira/browse/SPARK-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547379#comment-14547379 ] Ankur Chauhan commented on SPARK-6707: -- I had not thought of that. Plus, this is to

[jira] [Assigned] (SPARK-7691) Use type-specific row accessor functions in CatalystTypeConverters' toScala functions

[ https://issues.apache.org/jira/browse/SPARK-7691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7691: --- Assignee: Josh Rosen (was: Apache Spark) Use type-specific row accessor functions in

[jira] [Commented] (SPARK-7691) Use type-specific row accessor functions in CatalystTypeConverters' toScala functions

[ https://issues.apache.org/jira/browse/SPARK-7691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547383#comment-14547383 ] Apache Spark commented on SPARK-7691: - User 'JoshRosen' has created a pull request for

[jira] [Assigned] (SPARK-7691) Use type-specific row accessor functions in CatalystTypeConverters' toScala functions

[ https://issues.apache.org/jira/browse/SPARK-7691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7691: --- Assignee: Apache Spark (was: Josh Rosen) Use type-specific row accessor functions in

[jira] [Resolved] (SPARK-6514) For Kinesis Streaming, use the same region for DynamoDB (KCL checkpoints) as the Kinesis stream itself

[ https://issues.apache.org/jira/browse/SPARK-6514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-6514. -- Resolution: Fixed Fix Version/s: 1.4.0 For Kinesis Streaming, use the same region for

[jira] [Updated] (SPARK-6514) For Kinesis Streaming, use the same region for DynamoDB (KCL checkpoints) as the Kinesis stream itself

[ https://issues.apache.org/jira/browse/SPARK-6514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-6514: - Assignee: Chris Fregly For Kinesis Streaming, use the same region for DynamoDB (KCL checkpoints)

[jira] [Resolved] (SPARK-5960) Allow AWS credentials to be passed to KinesisUtils.createStream()

[ https://issues.apache.org/jira/browse/SPARK-5960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-5960. -- Resolution: Fixed Fix Version/s: 1.4.0 Allow AWS credentials to be passed to

[jira] [Resolved] (SPARK-7679) Update AWS SDK and KCL versions to 1.2.1

[ https://issues.apache.org/jira/browse/SPARK-7679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-7679. -- Resolution: Fixed Fix Version/s: 1.4.0 Update AWS SDK and KCL versions to 1.2.1

[jira] [Created] (SPARK-7692) Converst Kinesis examples to use new API instead of deprecated ones

Tathagata Das created SPARK-7692: Summary: Converst Kinesis examples to use new API instead of deprecated ones Key: SPARK-7692 URL: https://issues.apache.org/jira/browse/SPARK-7692 Project: Spark

[jira] [Resolved] (SPARK-6656) Allow the application name to be passed in versus pulling from SparkContext.getAppName()

[ https://issues.apache.org/jira/browse/SPARK-6656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-6656. -- Resolution: Fixed Fix Version/s: 1.4.0 Allow the application name to be passed in

[jira] [Created] (SPARK-7694) Use getOrElse for getting the threshold of LR model

Shuo Xiang created SPARK-7694: - Summary: Use getOrElse for getting the threshold of LR model Key: SPARK-7694 URL: https://issues.apache.org/jira/browse/SPARK-7694 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-7694) Use getOrElse for getting the threshold of LR model

[ https://issues.apache.org/jira/browse/SPARK-7694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7694: --- Assignee: Apache Spark Use getOrElse for getting the threshold of LR model

[jira] [Commented] (SPARK-7694) Use getOrElse for getting the threshold of LR model

[ https://issues.apache.org/jira/browse/SPARK-7694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547407#comment-14547407 ] Apache Spark commented on SPARK-7694: - User 'coderxiang' has created a pull request

[jira] [Updated] (SPARK-7694) Use getOrElse for getting the threshold of LR model

[ https://issues.apache.org/jira/browse/SPARK-7694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuo Xiang updated SPARK-7694: -- Description: The toString method of LogisticRegressionModel calls get method on an Option (threshold)

[jira] [Assigned] (SPARK-7693) Remove import scala.concurrent.ExecutionContext.Implicits.global

[ https://issues.apache.org/jira/browse/SPARK-7693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7693: --- Assignee: Apache Spark Remove import scala.concurrent.ExecutionContext.Implicits.global

[jira] [Commented] (SPARK-7693) Remove import scala.concurrent.ExecutionContext.Implicits.global

[ https://issues.apache.org/jira/browse/SPARK-7693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547406#comment-14547406 ] Apache Spark commented on SPARK-7693: - User 'zsxwing' has created a pull request for

[jira] [Assigned] (SPARK-7693) Remove import scala.concurrent.ExecutionContext.Implicits.global

[ https://issues.apache.org/jira/browse/SPARK-7693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7693: --- Assignee: (was: Apache Spark) Remove import

[jira] [Assigned] (SPARK-7673) DataSourceStrategy's buildPartitionedTableScan always list list file status for all data files

[ https://issues.apache.org/jira/browse/SPARK-7673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7673: --- Assignee: Cheng Lian (was: Apache Spark) DataSourceStrategy's buildPartitionedTableScan

[jira] [Commented] (SPARK-7673) DataSourceStrategy's buildPartitionedTableScan always list list file status for all data files

[ https://issues.apache.org/jira/browse/SPARK-7673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547450#comment-14547450 ] Apache Spark commented on SPARK-7673: - User 'liancheng' has created a pull request for

[jira] [Assigned] (SPARK-7673) DataSourceStrategy's buildPartitionedTableScan always list list file status for all data files

[ https://issues.apache.org/jira/browse/SPARK-7673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7673: --- Assignee: Apache Spark (was: Cheng Lian) DataSourceStrategy's buildPartitionedTableScan

[jira] [Created] (SPARK-7696) Aggregate function's result should be nullable only if the input expression is nullable

Haopu Wang created SPARK-7696: - Summary: Aggregate function's result should be nullable only if the input expression is nullable Key: SPARK-7696 URL: https://issues.apache.org/jira/browse/SPARK-7696

[jira] [Created] (SPARK-7698) Implement buffer pooling / re-use in ExecutorMemoryManager when using HeapAllocator

Josh Rosen created SPARK-7698: - Summary: Implement buffer pooling / re-use in ExecutorMemoryManager when using HeapAllocator Key: SPARK-7698 URL: https://issues.apache.org/jira/browse/SPARK-7698 Project:

[jira] [Assigned] (SPARK-7698) Implement buffer pooling / re-use in ExecutorMemoryManager when using HeapAllocator

[ https://issues.apache.org/jira/browse/SPARK-7698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7698: --- Assignee: Apache Spark (was: Josh Rosen) Implement buffer pooling / re-use in

[jira] [Assigned] (SPARK-7698) Implement buffer pooling / re-use in ExecutorMemoryManager when using HeapAllocator

[ https://issues.apache.org/jira/browse/SPARK-7698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7698: --- Assignee: Josh Rosen (was: Apache Spark) Implement buffer pooling / re-use in

[jira] [Commented] (SPARK-7275) Make LogicalRelation public

[ https://issues.apache.org/jira/browse/SPARK-7275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547521#comment-14547521 ] Glenn Weidner commented on SPARK-7275: -- I can look into making changes as suggested

[jira] [Created] (SPARK-7693) Remove import scala.concurrent.ExecutionContext.Implicits.global

Shixiong Zhu created SPARK-7693: --- Summary: Remove import scala.concurrent.ExecutionContext.Implicits.global Key: SPARK-7693 URL: https://issues.apache.org/jira/browse/SPARK-7693 Project: Spark

[jira] [Updated] (SPARK-7694) Use getOrElse for getting the threshold of LR model

[ https://issues.apache.org/jira/browse/SPARK-7694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuo Xiang updated SPARK-7694: -- Description: The toString method of LogisticRegressionModel calls get method on an Option (threshold)

[jira] [Commented] (SPARK-5711) Sort Shuffle performance issues about using AppendOnlyMap for large data sets

[ https://issues.apache.org/jira/browse/SPARK-5711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547433#comment-14547433 ] Sun Fulin commented on SPARK-5711: -- [~srowen] After changing from spark 1.2.0 to 1.3.0

[jira] [Commented] (SPARK-7663) [MLLIB] feature.Word2Vec throws empty iterator error when the vocabulary size is zero

[ https://issues.apache.org/jira/browse/SPARK-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547466#comment-14547466 ] Xusen Yin commented on SPARK-7663: -- Got it. Thanks Sean. [MLLIB] feature.Word2Vec

[jira] [Created] (SPARK-7697) Column with an unsigned int should be treated as long in JDBCRDD

DAITO Teppei created SPARK-7697: --- Summary: Column with an unsigned int should be treated as long in JDBCRDD Key: SPARK-7697 URL: https://issues.apache.org/jira/browse/SPARK-7697 Project: Spark

[jira] [Updated] (SPARK-7687) DataFrame.describe() should cast all aggregates to doubles

[ https://issues.apache.org/jira/browse/SPARK-7687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-7687: -- Description: In DataFrame.describe(), the count aggregate produces an integer, the avg and stdev

[jira] [Updated] (SPARK-7687) DataFrame.describe() should cast all aggregates to String

[ https://issues.apache.org/jira/browse/SPARK-7687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-7687: -- Summary: DataFrame.describe() should cast all aggregates to String (was: DataFrame.describe() should

[jira] [Assigned] (SPARK-7694) Use getOrElse for getting the threshold of LR model

[ https://issues.apache.org/jira/browse/SPARK-7694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7694: --- Assignee: (was: Apache Spark) Use getOrElse for getting the threshold of LR model

[jira] [Updated] (SPARK-7694) Use getOrElse for getting the threshold of LR model

[ https://issues.apache.org/jira/browse/SPARK-7694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuo Xiang updated SPARK-7694: -- Description: The toString method of LogisticRegressionModel calls get method on an Option (threshold)

[jira] [Updated] (SPARK-7694) Use getOrElse for getting the threshold of LR model

[ https://issues.apache.org/jira/browse/SPARK-7694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuo Xiang updated SPARK-7694: -- Description: The toString method of LogisticRegressionModel calls get method on an Option (threshold)

[jira] [Updated] (SPARK-7695) Investigate use of checkerframework.org annotations processor / static analysis library

[ https://issues.apache.org/jira/browse/SPARK-7695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-7695: -- Description: For Project Tungsten, I'd like to investigate the use of the University of Washington's

[jira] [Created] (SPARK-7695) Investigate use of checkerframework.org annotations processor / static analysis library

Josh Rosen created SPARK-7695: - Summary: Investigate use of checkerframework.org annotations processor / static analysis library Key: SPARK-7695 URL: https://issues.apache.org/jira/browse/SPARK-7695

[jira] [Commented] (SPARK-7275) Make LogicalRelation public

[ https://issues.apache.org/jira/browse/SPARK-7275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547510#comment-14547510 ] Reynold Xin commented on SPARK-7275: I think we can move everything currently in

[jira] [Updated] (SPARK-7687) DataFrame.describe() should cast all aggregates to String

[ https://issues.apache.org/jira/browse/SPARK-7687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-7687: -- Description: In DataFrame.describe(), the count aggregate produces an integer, the avg and stdev

[jira] [Commented] (SPARK-6707) Mesos Scheduler should allow the user to specify constraints based on slave attributes

[ https://issues.apache.org/jira/browse/SPARK-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547357#comment-14547357 ] Eron Wright commented on SPARK-6707: - A related enhancement would be to allow the

[jira] [Commented] (SPARK-6657) Fix Python doc build warnings

[ https://issues.apache.org/jira/browse/SPARK-6657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547359#comment-14547359 ] Apache Spark commented on SPARK-6657: - User 'mengxr' has created a pull request for

[jira] [Assigned] (SPARK-6657) Fix Python doc build warnings

[ https://issues.apache.org/jira/browse/SPARK-6657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-6657: Assignee: Xiangrui Meng Fix Python doc build warnings -

[jira] [Commented] (SPARK-7275) Make LogicalRelation public

[ https://issues.apache.org/jira/browse/SPARK-7275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547362#comment-14547362 ] Santiago M. Mola commented on SPARK-7275: - [~rxin] What are your thoughts on this?

[jira] [Commented] (SPARK-6246) spark-ec2 can't handle clusters with 100 nodes

[ https://issues.apache.org/jira/browse/SPARK-6246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547501#comment-14547501 ] Shivaram Venkataraman commented on SPARK-6246: -- I just ran into this problem

[jira] [Closed] (SPARK-7694) Use getOrElse for getting the threshold of LR model

[ https://issues.apache.org/jira/browse/SPARK-7694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng closed SPARK-7694. Resolution: Fixed Fix Version/s: 1.4.0 Assignee: Shuo Xiang Target

[jira] [Commented] (SPARK-4823) rowSimilarities

[ https://issues.apache.org/jira/browse/SPARK-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547318#comment-14547318 ] Debasish Das commented on SPARK-4823: - I opened up a PR that worked well for our

[jira] [Assigned] (SPARK-7272) User guide update for PMML model export

[ https://issues.apache.org/jira/browse/SPARK-7272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7272: --- Assignee: Vincenzo Selvaggio (was: Apache Spark) User guide update for PMML model export

[jira] [Assigned] (SPARK-7272) User guide update for PMML model export

[ https://issues.apache.org/jira/browse/SPARK-7272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7272: --- Assignee: Apache Spark (was: Vincenzo Selvaggio) User guide update for PMML model export

[jira] [Commented] (SPARK-6416) RDD.fold() requires the operator to be commutative

[ https://issues.apache.org/jira/browse/SPARK-6416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547120#comment-14547120 ] Sean Owen commented on SPARK-6416: -- Josh I'm looking at the related SPARK-7683 which will

[jira] [Commented] (SPARK-7272) User guide update for PMML model export

[ https://issues.apache.org/jira/browse/SPARK-7272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547148#comment-14547148 ] Vincenzo Selvaggio commented on SPARK-7272: --- [~mengxr] [~josephkb] Please review

[jira] [Assigned] (SPARK-7689) Deprecate spark.cleaner.ttl

[ https://issues.apache.org/jira/browse/SPARK-7689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-7689: - Assignee: Josh Rosen Deprecate spark.cleaner.ttl ---

[jira] [Assigned] (SPARK-7690) MulticlassClassificationEvaluator for tuning Multiclass Classifiers

[ https://issues.apache.org/jira/browse/SPARK-7690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ram Sriharsha reassigned SPARK-7690: Assignee: Ram Sriharsha MulticlassClassificationEvaluator for tuning Multiclass

[jira] [Created] (SPARK-7690) MulticlassClassificationEvaluator for tuning Multiclass Classifiers

Ram Sriharsha created SPARK-7690: Summary: MulticlassClassificationEvaluator for tuning Multiclass Classifiers Key: SPARK-7690 URL: https://issues.apache.org/jira/browse/SPARK-7690 Project: Spark

[jira] [Updated] (SPARK-7689) Deprecate spark.cleaner.ttl

[ https://issues.apache.org/jira/browse/SPARK-7689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-7689: -- Target Version/s: 1.4.0 Deprecate spark.cleaner.ttl ---

[jira] [Created] (SPARK-7691) Use type-specific row accessor functions in CatalystTypeConverters' toScala functions

Josh Rosen created SPARK-7691: - Summary: Use type-specific row accessor functions in CatalystTypeConverters' toScala functions Key: SPARK-7691 URL: https://issues.apache.org/jira/browse/SPARK-7691

[jira] [Resolved] (SPARK-7686) Runnable DescribeCommand is assigned wrong physical plan output attributes in SparkStrategies

[ https://issues.apache.org/jira/browse/SPARK-7686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-7686. Resolution: Fixed Fix Version/s: 1.4.0 Target Version/s: 1.4.0 (was: 1.3.2, 1.4.0)

[jira] [Resolved] (SPARK-7491) Handle drivers for Metastore JDBC

[ https://issues.apache.org/jira/browse/SPARK-7491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-7491. - Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 6167

[jira] [Updated] (SPARK-7491) Handle drivers for Metastore JDBC

[ https://issues.apache.org/jira/browse/SPARK-7491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-7491: Assignee: Michael Armbrust Handle drivers for Metastore JDBC

[jira] [Commented] (SPARK-7688) PySpark + ipython throws port out of range exception

[ https://issues.apache.org/jira/browse/SPARK-7688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547313#comment-14547313 ] Davies Liu commented on SPARK-7688: --- It runs fine in my Mac, could you try this? {code}

[jira] [Resolved] (SPARK-7447) Large Job submission lag when using Parquet w/ Schema Merging

[ https://issues.apache.org/jira/browse/SPARK-7447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-7447. --- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 6012

[jira] [Commented] (SPARK-7540) PMML correctness check

[ https://issues.apache.org/jira/browse/SPARK-7540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547056#comment-14547056 ] Vincenzo Selvaggio commented on SPARK-7540: --- All models supporting the pmml

[jira] [Commented] (SPARK-7654) DataFrameReader and DataFrameWriter for input/output API

[ https://issues.apache.org/jira/browse/SPARK-7654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547070#comment-14547070 ] Apache Spark commented on SPARK-7654: - User 'rxin' has created a pull request for this

  1   2   >