[jira] [Updated] (SPARK-10981) R semijoin leads to Java errors, R leftsemi leads to Spark errors

2015-10-07 Thread Monica Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Monica Liu updated SPARK-10981: --- Description: I am using SparkR from RStudio, and I ran into an error with the join function that I

[jira] [Updated] (SPARK-10909) Spark sql jdbc fails for Oracle NUMBER type columns

2015-10-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-10909: Labels: (was: jdbc newbie sql) > Spark sql jdbc fails for Oracle NUMBER type columns >

[jira] [Updated] (SPARK-10956) Introduce common memory management interface for execution and storage

2015-10-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10956: -- Affects Version/s: (was: 1.0.0) > Introduce common memory management interface for execution and

[jira] [Resolved] (SPARK-10941) .Refactor AggregateFunction2 and AlgebraicAggregate interfaces to improve code clarity

2015-10-07 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-10941. -- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8973

[jira] [Created] (SPARK-10981) R semijoin leads to Java errors, R leftsemi leads to Spark errors

2015-10-07 Thread Monica Liu (JIRA)
Monica Liu created SPARK-10981: -- Summary: R semijoin leads to Java errors, R leftsemi leads to Spark errors Key: SPARK-10981 URL: https://issues.apache.org/jira/browse/SPARK-10981 Project: Spark

[jira] [Updated] (SPARK-10000) Consolidate cache memory management and execution memory management

2015-10-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1: -- Description: Memory management in Spark is currently broken down into two disjoint regions: one for

[jira] [Updated] (SPARK-10000) Consolidate cache memory management and execution memory management

2015-10-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1: -- Attachment: unified-memory-management-spark-1.pdf > Consolidate cache memory management and

[jira] [Resolved] (SPARK-10679) javax.jdo.JDOFatalUserException in executor

2015-10-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-10679. - Resolution: Fixed Fix Version/s: 1.6.0 > javax.jdo.JDOFatalUserException in executor >

[jira] [Updated] (SPARK-10914) Incorrect empty join sets when executor-memory >= 32g

2015-10-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-10914: Description: Using an inner join, to match together two integer columns, I generally get no

[jira] [Commented] (SPARK-10914) Incorrect empty join sets when executor-memory >= 32g

2015-10-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947711#comment-14947711 ] Reynold Xin commented on SPARK-10914: - I don't think size estimator would impact the result. If I

[jira] [Resolved] (SPARK-10982) Rename ExpressionAggregate -> DeclarativeAggregate

2015-10-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-10982. Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9013

[jira] [Updated] (SPARK-10490) Consolidate the Cholesky solvers in WeightedLeastSquares and ALS

2015-10-07 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10490: -- Assignee: Yanbo Liang > Consolidate the Cholesky solvers in WeightedLeastSquares and ALS >

[jira] [Updated] (SPARK-10875) RowMatrix.computeCovariance() result is not exactly symmetric

2015-10-07 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10875: -- Shepherd: Xiangrui Meng Target Version/s: 1.6.0 > RowMatrix.computeCovariance()

[jira] [Commented] (SPARK-10987) yarn-cluster mode misbehaving with netty-based RPC backend

2015-10-07 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947788#comment-14947788 ] Marcelo Vanzin commented on SPARK-10987: Hmm. I think I know what's going on, just not how. In

[jira] [Created] (SPARK-10985) Avoid passing evicted blocks throughout BlockManager / CacheManager

2015-10-07 Thread Andrew Or (JIRA)
Andrew Or created SPARK-10985: - Summary: Avoid passing evicted blocks throughout BlockManager / CacheManager Key: SPARK-10985 URL: https://issues.apache.org/jira/browse/SPARK-10985 Project: Spark

[jira] [Updated] (SPARK-7869) Spark Data Frame Fails to Load Postgres Tables with JSONB DataType Columns

2015-10-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7869: --- Assignee: Alexey Grishchenko > Spark Data Frame Fails to Load Postgres Tables with JSONB DataType

[jira] [Updated] (SPARK-7869) Spark Data Frame Fails to Load Postgres Tables with JSONB DataType Columns

2015-10-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7869: --- Target Version/s: 1.6.0 > Spark Data Frame Fails to Load Postgres Tables with JSONB DataType Columns

[jira] [Updated] (SPARK-10986) ClassNotFoundException when running on Client mode, with a Mesos master.

2015-10-07 Thread Joseph Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Wu updated SPARK-10986: -- Description: When running an example task on a Mesos cluster (local master, local agent), any Spark

[jira] [Created] (SPARK-10986) ClassNotFoundException when running on Client mode, with a Mesos master.

2015-10-07 Thread Joseph Wu (JIRA)
Joseph Wu created SPARK-10986: - Summary: ClassNotFoundException when running on Client mode, with a Mesos master. Key: SPARK-10986 URL: https://issues.apache.org/jira/browse/SPARK-10986 Project: Spark

[jira] [Commented] (SPARK-8386) DataFrame and JDBC regression

2015-10-07 Thread Huaxin Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947749#comment-14947749 ] Huaxin Gao commented on SPARK-8386: --- If the above fix is correct, can I have a pull request to check in

[jira] [Updated] (SPARK-9702) Repartition operator should use Exchange to perform its shuffle

2015-10-07 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-9702: Assignee: Josh Rosen > Repartition operator should use Exchange to perform its shuffle >

[jira] [Commented] (SPARK-10000) Consolidate cache memory management and execution memory management

2015-10-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947530#comment-14947530 ] Reynold Xin commented on SPARK-1: - [~bowenzhangusa] thanks for the interest. This task is pretty

[jira] [Commented] (SPARK-10000) Consolidate cache memory management and execution memory management

2015-10-07 Thread Bowen Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947558#comment-14947558 ] Bowen Zhang commented on SPARK-1: - [~rxin], sounds good. > Consolidate cache memory management

[jira] [Commented] (SPARK-10987) yarn-cluster mode misbehaving with netty-based RPC backend

2015-10-07 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947738#comment-14947738 ] Marcelo Vanzin commented on SPARK-10987: It may not be cluster mode per se; I ran the tests

[jira] [Updated] (SPARK-10000) Consolidate storage and execution memory management

2015-10-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1: -- Summary: Consolidate storage and execution memory management (was: Consolidate cache memory

[jira] [Updated] (SPARK-10980) Create wrong decimal if unscaled > 1e18 and scale > 0

2015-10-07 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-10980: --- Description: Decimal(100L, 20, 2) will become 100 instead of

[jira] [Resolved] (SPARK-10779) Set initialModel for KMeans model in PySpark (spark.mllib)

2015-10-07 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-10779. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8967

[jira] [Commented] (SPARK-10987) yarn-cluster mode misbehaving with netty-based RPC backend

2015-10-07 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947743#comment-14947743 ] Marcelo Vanzin commented on SPARK-10987: The {{ExecutorRunnable}} process (client-mode AM) was

[jira] [Resolved] (SPARK-10738) Refactoring `Instance` out from LOR and LIR, and also cleaning up some code

2015-10-07 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-10738. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8853

[jira] [Resolved] (SPARK-10917) Improve performance of complex types in columnar cache

2015-10-07 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-10917. Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8971

[jira] [Updated] (SPARK-10987) yarn-client mode misbehaving with netty-based RPC backend

2015-10-07 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-10987: --- Summary: yarn-client mode misbehaving with netty-based RPC backend (was: yarn-cluster mode

[jira] [Assigned] (SPARK-10980) Create wrong decimal if unscaled > 1e18 and scale > 0

2015-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10980: Assignee: Davies Liu (was: Apache Spark) > Create wrong decimal if unscaled > 1e18 and

[jira] [Commented] (SPARK-10980) Create wrong decimal if unscaled > 1e18 and scale > 0

2015-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947534#comment-14947534 ] Apache Spark commented on SPARK-10980: -- User 'davies' has created a pull request for this issue:

[jira] [Commented] (SPARK-10942) Not all cached RDDs are unpersisted

2015-10-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947533#comment-14947533 ] Sean Owen commented on SPARK-10942: --- I tried this on master in spark-shell: {code} import

[jira] [Assigned] (SPARK-10980) Create wrong decimal if unscaled > 1e18 and scale > 0

2015-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10980: Assignee: Apache Spark (was: Davies Liu) > Create wrong decimal if unscaled > 1e18 and

[jira] [Resolved] (SPARK-10300) Use tags to control which tests to run depending on changes being tested

2015-10-07 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-10300. Resolution: Fixed Second time is the charm? > Use tags to control which tests to run

[jira] [Updated] (SPARK-10000) Consolidate cache memory management and execution memory management

2015-10-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1: -- Description: Memory management in Spark is currently broken down into two disjoint regions: one for

[jira] [Comment Edited] (SPARK-10914) Incorrect empty join sets when executor-memory >= 32g

2015-10-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14945358#comment-14945358 ] Reynold Xin edited comment on SPARK-10914 at 10/7/15 10:22 PM: --- Thanks for

[jira] [Resolved] (SPARK-10490) Consolidate the Cholesky solvers in WeightedLeastSquares and ALS

2015-10-07 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-10490. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8936

[jira] [Created] (SPARK-10980) Create wrong decimal with unscaled value and precision > 18

2015-10-07 Thread Davies Liu (JIRA)
Davies Liu created SPARK-10980: -- Summary: Create wrong decimal with unscaled value and precision > 18 Key: SPARK-10980 URL: https://issues.apache.org/jira/browse/SPARK-10980 Project: Spark

[jira] [Updated] (SPARK-10980) Create wrong decimal with unscaled value and precision > 18

2015-10-07 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-10980: --- Affects Version/s: 1.2.2 1.3.1 1.4.1 > Create wrong

[jira] [Updated] (SPARK-10980) Create wrong decimal if unscaled > 1e18 and scale > 0

2015-10-07 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-10980: --- Summary: Create wrong decimal if unscaled > 1e18 and scale > 0 (was: Create wrong decimal with

[jira] [Commented] (SPARK-10342) Cooperative memory management

2015-10-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947687#comment-14947687 ] Reynold Xin commented on SPARK-10342: - [~fxing] thanks a lot for the interest. Since this is your

[jira] [Commented] (SPARK-8386) DataFrame and JDBC regression

2015-10-07 Thread Huaxin Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947736#comment-14947736 ] Huaxin Gao commented on SPARK-8386: --- I looked the code, it has this @deprecated("Use write.jdbc()",

[jira] [Updated] (SPARK-10959) PySpark StreamingLogisticRegressionWithSGD does not train with given regParam and convergenceTol parameters

2015-10-07 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10959: -- Assignee: Bryan Cutler > PySpark StreamingLogisticRegressionWithSGD does not train with given

[jira] [Resolved] (SPARK-10980) Create wrong decimal if unscaled > 1e18 and scale > 0

2015-10-07 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-10980. Resolution: Fixed Fix Version/s: 1.2.3 1.5.2 1.3.2

[jira] [Created] (SPARK-10982) Rename ExpressionAggregate -> DeclarativeAggregate

2015-10-07 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-10982: --- Summary: Rename ExpressionAggregate -> DeclarativeAggregate Key: SPARK-10982 URL: https://issues.apache.org/jira/browse/SPARK-10982 Project: Spark Issue Type:

[jira] [Updated] (SPARK-10956) Introduce common memory management interface for execution and storage

2015-10-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10956: -- Description: The first step towards implementing a solution for SPARK-1 is to refactor the

[jira] [Updated] (SPARK-10956) Introduce common memory management interface for execution and storage

2015-10-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10956: -- Description: The first step towards implementing a solution for SPARK-1 is to refactor the

[jira] [Created] (SPARK-10984) Simplify *MemoryManager class structure

2015-10-07 Thread Andrew Or (JIRA)
Andrew Or created SPARK-10984: - Summary: Simplify *MemoryManager class structure Key: SPARK-10984 URL: https://issues.apache.org/jira/browse/SPARK-10984 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-10956) Introduce common memory management interface for execution and storage

2015-10-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10956: -- Priority: Major (was: Critical) > Introduce common memory management interface for execution and

[jira] [Updated] (SPARK-10779) Set initialModel for KMeans model in PySpark (spark.mllib)

2015-10-07 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-10779: -- Assignee: Evan Chen > Set initialModel for KMeans model in PySpark (spark.mllib) >

[jira] [Comment Edited] (SPARK-10914) Incorrect empty join sets when executor-memory >= 32g

2015-10-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14945441#comment-14945441 ] Reynold Xin edited comment on SPARK-10914 at 10/7/15 10:12 PM: --- I just ran

[jira] [Updated] (SPARK-10914) Incorrect empty join sets when executor-memory >= 32g

2015-10-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-10914: Description: Using an inner join, to match together two integer columns, I generally get no

[jira] [Created] (SPARK-10987) yarn-cluster mode misbehaving with netty-based RPC backend

2015-10-07 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-10987: -- Summary: yarn-cluster mode misbehaving with netty-based RPC backend Key: SPARK-10987 URL: https://issues.apache.org/jira/browse/SPARK-10987 Project: Spark

[jira] [Updated] (SPARK-10919) Association rules class should return the support of each rule

2015-10-07 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-10919: -- Summary: Association rules class should return the support of each rule (was:

[jira] [Commented] (SPARK-10919) Assosiation rules class should return the support of each rule

2015-10-07 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947720#comment-14947720 ] Joseph K. Bradley commented on SPARK-10919: --- Would you be interested in sending a PR for this?

[jira] [Resolved] (SPARK-9702) Repartition operator should use Exchange to perform its shuffle

2015-10-07 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-9702. - Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8083

[jira] [Updated] (SPARK-10875) RowMatrix.computeCovariance() result is not exactly symmetric

2015-10-07 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10875: -- Assignee: Nick Pritchard > RowMatrix.computeCovariance() result is not exactly symmetric >

[jira] [Commented] (SPARK-10982) Rename ExpressionAggregate -> DeclarativeAggregate

2015-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947526#comment-14947526 ] Apache Spark commented on SPARK-10982: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10982) Rename ExpressionAggregate -> DeclarativeAggregate

2015-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10982: Assignee: Reynold Xin (was: Apache Spark) > Rename ExpressionAggregate ->

[jira] [Assigned] (SPARK-10982) Rename ExpressionAggregate -> DeclarativeAggregate

2015-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10982: Assignee: Apache Spark (was: Reynold Xin) > Rename ExpressionAggregate ->

[jira] [Created] (SPARK-10983) Implement unified memory manager

2015-10-07 Thread Andrew Or (JIRA)
Andrew Or created SPARK-10983: - Summary: Implement unified memory manager Key: SPARK-10983 URL: https://issues.apache.org/jira/browse/SPARK-10983 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-10856) SQL Server dialect needs to map java.sql.Timestamp to DATETIME instead of TIMESTAMP

2015-10-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-10856. - Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 1.6.0 > SQL Server

[jira] [Updated] (SPARK-7869) Spark Data Frame Fails to Load Postgres Tables with JSONB DataType Columns

2015-10-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7869: --- Description: Most of our tables load into dataframes just fine with postgres. However we have a

[jira] [Updated] (SPARK-10186) Add support for more postgres column types

2015-10-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-10186: Labels: (was: array json postgres sql struct) > Add support for more postgres column types >

[jira] [Updated] (SPARK-10959) PySpark StreamingLogisticRegressionWithSGD does not train with given regParam and convergenceTol parameters

2015-10-07 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10959: -- Target Version/s: 1.5.2, 1.6.0 > PySpark StreamingLogisticRegressionWithSGD does not train

[jira] [Updated] (SPARK-10959) PySpark StreamingLogisticRegressionWithSGD does not train with given regParam and convergenceTol parameters

2015-10-07 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10959: -- Shepherd: Xiangrui Meng > PySpark StreamingLogisticRegressionWithSGD does not train with given

[jira] [Commented] (SPARK-8386) DataFrame and JDBC regression

2015-10-07 Thread Huaxin Gao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947775#comment-14947775 ] Huaxin Gao commented on SPARK-8386: --- Actually I can also recreate the problem in the other two cases.

[jira] [Created] (SPARK-10988) Reduce duplication in Aggregate2's expression rewriting logic

2015-10-07 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-10988: -- Summary: Reduce duplication in Aggregate2's expression rewriting logic Key: SPARK-10988 URL: https://issues.apache.org/jira/browse/SPARK-10988 Project: Spark

[jira] [Updated] (SPARK-10967) Incorrect Join behavior in filter conditions

2015-10-07 Thread RaviShankar KS (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] RaviShankar KS updated SPARK-10967: --- Attachment: CreateDF_sparkshell_jira.scala run in spark shell > Incorrect Join behavior in

[jira] [Updated] (SPARK-10968) Incorrect Join behavior in filter conditions

2015-10-07 Thread RaviShankar KS (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] RaviShankar KS updated SPARK-10968: --- Attachment: CreateDF_sparkshell_jira.scala > Incorrect Join behavior in filter conditions >

[jira] [Commented] (SPARK-10965) Optimize filesEqualRecursive

2015-10-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14946469#comment-14946469 ] Sean Owen commented on SPARK-10965: --- You don't need it to be assigned to you, just go ahead. I will add

[jira] [Updated] (SPARK-10967) Incorrect Join behavior in filter conditions

2015-10-07 Thread RaviShankar KS (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] RaviShankar KS updated SPARK-10967: --- Description: We notice that the join conditions are not working as expected in the case of

[jira] [Closed] (SPARK-10967) ignore - Incorrect UNION ALL behavior

2015-10-07 Thread RaviShankar KS (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] RaviShankar KS closed SPARK-10967. -- Resolution: Invalid Target Version/s: 1.5.1, 1.4.1 (was: 1.4.1, 1.5.1) > ignore -

[jira] [Updated] (SPARK-10967) Incorrect UNION ALL behavior

2015-10-07 Thread RaviShankar KS (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] RaviShankar KS updated SPARK-10967: --- Attachment: (was: CreateDF_sparkshell_jira.scala) > Incorrect UNION ALL behavior >

[jira] [Updated] (SPARK-10967) Incorrect UNION ALL behavior

2015-10-07 Thread RaviShankar KS (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] RaviShankar KS updated SPARK-10967: --- Description: IGNORE (was: We notice that the join conditions are not working as expected in

[jira] [Updated] (SPARK-10967) ignore - Incorrect UNION ALL behavior

2015-10-07 Thread RaviShankar KS (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] RaviShankar KS updated SPARK-10967: --- Summary: ignore - Incorrect UNION ALL behavior (was: Incorrect UNION ALL behavior) >

[jira] [Commented] (SPARK-10968) Incorrect Join behavior in filter conditions

2015-10-07 Thread RaviShankar KS (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14946493#comment-14946493 ] RaviShankar KS commented on SPARK-10968: not actually incorrect. DataFrame d5.value has fields

[jira] [Commented] (SPARK-10968) Incorrect Join behavior in filter conditions

2015-10-07 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14946358#comment-14946358 ] Liang-Chi Hsieh commented on SPARK-10968: - Is it incorrect? Because d5.value and d5_opp.value are

[jira] [Assigned] (SPARK-10988) Reduce duplication in Aggregate2's expression rewriting logic

2015-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10988: Assignee: Apache Spark (was: Josh Rosen) > Reduce duplication in Aggregate2's expression

[jira] [Resolved] (SPARK-10064) Decision tree continuous feature binning is slow in large feature spaces

2015-10-07 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-10064. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8246

[jira] [Assigned] (SPARK-10992) Partial Aggregation Support for Hive UDAF

2015-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10992: Assignee: Apache Spark > Partial Aggregation Support for Hive UDAF >

[jira] [Commented] (SPARK-10992) Partial Aggregation Support for Hive UDAF

2015-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947912#comment-14947912 ] Apache Spark commented on SPARK-10992: -- User 'chenghao-intel' has created a pull request for this

[jira] [Assigned] (SPARK-10992) Partial Aggregation Support for Hive UDAF

2015-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10992: Assignee: (was: Apache Spark) > Partial Aggregation Support for Hive UDAF >

[jira] [Created] (SPARK-10997) Netty-based RPC env should support a "client-only" mode.

2015-10-07 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-10997: -- Summary: Netty-based RPC env should support a "client-only" mode. Key: SPARK-10997 URL: https://issues.apache.org/jira/browse/SPARK-10997 Project: Spark

[jira] [Commented] (SPARK-8386) DataFrame and JDBC regression

2015-10-07 Thread Peter Haumer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947815#comment-14947815 ] Peter Haumer commented on SPARK-8386: - Huaxin Gao, sorry for not replying earlier. It slipped through

[jira] [Commented] (SPARK-10988) Reduce duplication in Aggregate2's expression rewriting logic

2015-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947817#comment-14947817 ] Apache Spark commented on SPARK-10988: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10988) Reduce duplication in Aggregate2's expression rewriting logic

2015-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10988: Assignee: Josh Rosen (was: Apache Spark) > Reduce duplication in Aggregate2's expression

[jira] [Commented] (SPARK-10940) Too many open files Spark Shuffle

2015-10-07 Thread Sandeep Pal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947836#comment-14947836 ] Sandeep Pal commented on SPARK-10940: - I rebooted all the machines and issue is not reproduced.

[jira] [Commented] (SPARK-10990) Avoid the serialization multiple times during unrolling of complex types

2015-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947838#comment-14947838 ] Apache Spark commented on SPARK-10990: -- User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10990) Avoid the serialization multiple times during unrolling of complex types

2015-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10990: Assignee: Davies Liu (was: Apache Spark) > Avoid the serialization multiple times during

[jira] [Created] (SPARK-10990) Avoid the serialization multiple times during unrolling of complex types

2015-10-07 Thread Davies Liu (JIRA)
Davies Liu created SPARK-10990: -- Summary: Avoid the serialization multiple times during unrolling of complex types Key: SPARK-10990 URL: https://issues.apache.org/jira/browse/SPARK-10990 Project: Spark

[jira] [Commented] (SPARK-10987) yarn-client mode misbehaving with netty-based RPC backend

2015-10-07 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947893#comment-14947893 ] Marcelo Vanzin commented on SPARK-10987: Anyway, here's what I found so far. Driver launches AM;

[jira] [Assigned] (SPARK-10989) Add the dot and hadamard products to the Vectors object

2015-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10989: Assignee: Apache Spark > Add the dot and hadamard products to the Vectors object >

[jira] [Created] (SPARK-10994) Local clustering coefficient computation in GraphX

2015-10-07 Thread Yang Yang (JIRA)
Yang Yang created SPARK-10994: - Summary: Local clustering coefficient computation in GraphX Key: SPARK-10994 URL: https://issues.apache.org/jira/browse/SPARK-10994 Project: Spark Issue Type: New

[jira] [Assigned] (SPARK-10989) Add the dot and hadamard products to the Vectors object

2015-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10989: Assignee: (was: Apache Spark) > Add the dot and hadamard products to the Vectors

[jira] [Commented] (SPARK-10998) Show non-children in default Expression.toString

2015-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14948007#comment-14948007 ] Apache Spark commented on SPARK-10998: -- User 'marmbrus' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10998) Show non-children in default Expression.toString

2015-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10998: Assignee: Michael Armbrust (was: Apache Spark) > Show non-children in default

[jira] [Assigned] (SPARK-10767) Make pyspark shared params codegen more consistent

2015-10-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10767: Assignee: (was: Apache Spark) > Make pyspark shared params codegen more consistent >

  1   2   >