[jira] [Created] (SPARK-18484) case class datasets - ability to specify decimal precision and scale

2016-11-16 Thread Damian Momot (JIRA)
Damian Momot created SPARK-18484: Summary: case class datasets - ability to specify decimal precision and scale Key: SPARK-18484 URL: https://issues.apache.org/jira/browse/SPARK-18484 Project: Spark

[jira] [Updated] (SPARK-18484) case class datasets - ability to specify decimal precision and scale

2016-11-16 Thread Damian Momot (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Momot updated SPARK-18484: - Description: Currently when using decimal type (BigDecimal in scala case class) there's no way t

[jira] [Commented] (SPARK-18478) Support codegen for Hive UDFs

2016-11-16 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15673032#comment-15673032 ] Takeshi Yamamuro commented on SPARK-18478: -- ya, I'll make a pr later, thanks! >

[jira] [Commented] (SPARK-18478) Support codegen for Hive UDFs

2016-11-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15673027#comment-15673027 ] Reynold Xin commented on SPARK-18478: - Yea that it seems like it's worth doing. > S

[jira] [Commented] (SPARK-18478) Support codegen for Hive UDFs

2016-11-16 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15673017#comment-15673017 ] Takeshi Yamamuro commented on SPARK-18478: -- [~rxin] seems we have some performan

[jira] [Commented] (SPARK-14974) spark sql job create too many files in HDFS when doing insert overwrite hive table

2016-11-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15672948#comment-15672948 ] Apache Spark commented on SPARK-14974: -- User 'baishuo' has created a pull request fo

[jira] [Commented] (SPARK-18074) UDFs don't work on non-local environment

2016-11-16 Thread roncenzhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15672950#comment-15672950 ] roncenzhao commented on SPARK-18074: I have encountered this problem, too. If any one

[jira] [Created] (SPARK-18483) spark on yarn always connect to yarn resourcemanager at 0.0.0.0:8032

2016-11-16 Thread inred (JIRA)
inred created SPARK-18483: - Summary: spark on yarn always connect to yarn resourcemanager at 0.0.0.0:8032 Key: SPARK-18483 URL: https://issues.apache.org/jira/browse/SPARK-18483 Project: Spark Iss

[jira] [Commented] (SPARK-17662) Dedup UDAF

2016-11-16 Thread Ohad Raviv (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15672928#comment-15672928 ] Ohad Raviv commented on SPARK-17662: you're right, great solution! I didn't know abou

[jira] [Commented] (SPARK-18470) Provide Spark Streaming Monitor Rest Api

2016-11-16 Thread Genmao Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15672745#comment-15672745 ] Genmao Yu commented on SPARK-18470: --- Thanks for your suggestions, i will provide it lat

[jira] [Commented] (SPARK-18481) ML 2.1 QA: Remove deprecated methods for ML

2016-11-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15672708#comment-15672708 ] Apache Spark commented on SPARK-18481: -- User 'yanboliang' has created a pull request

[jira] [Created] (SPARK-18482) make sure Spark can access the table metadata created by older version of spark

2016-11-16 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-18482: --- Summary: make sure Spark can access the table metadata created by older version of spark Key: SPARK-18482 URL: https://issues.apache.org/jira/browse/SPARK-18482 Project

[jira] [Updated] (SPARK-1267) Add a pip installer for PySpark

2016-11-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-1267: -- Fix Version/s: 2.1.0 > Add a pip installer for PySpark > --- > >

[jira] [Updated] (SPARK-18129) Sign pip artifacts

2016-11-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-18129: --- Fix Version/s: 2.1.0 > Sign pip artifacts > -- > > Key: SPARK-18129 >

[jira] [Created] (SPARK-18481) ML 2.1 QA: Remove deprecated methods for ML

2016-11-16 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-18481: --- Summary: ML 2.1 QA: Remove deprecated methods for ML Key: SPARK-18481 URL: https://issues.apache.org/jira/browse/SPARK-18481 Project: Spark Issue Type: Improv

[jira] [Resolved] (SPARK-18442) Fix nullability of WrapOption.

2016-11-16 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-18442. - Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 15887 [https://githu

[jira] [Updated] (SPARK-18442) Fix nullability of WrapOption.

2016-11-16 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-18442: Assignee: Takuya Ueshin > Fix nullability of WrapOption. > -- > >

[jira] [Assigned] (SPARK-18480) Link validation for ML guides

2016-11-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18480: Assignee: (was: Apache Spark) > Link validation for ML guides > --

[jira] [Assigned] (SPARK-18480) Link validation for ML guides

2016-11-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18480: Assignee: Apache Spark > Link validation for ML guides > - > >

[jira] [Commented] (SPARK-18480) Link validation for ML guides

2016-11-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15672502#comment-15672502 ] Apache Spark commented on SPARK-18480: -- User 'zhengruifeng' has created a pull reque

[jira] [Created] (SPARK-18480) Link validation for ML guides

2016-11-16 Thread zhengruifeng (JIRA)
zhengruifeng created SPARK-18480: Summary: Link validation for ML guides Key: SPARK-18480 URL: https://issues.apache.org/jira/browse/SPARK-18480 Project: Spark Issue Type: Bug Compo

[jira] [Commented] (SPARK-18478) Support codegen for Hive UDFs

2016-11-16 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15672493#comment-15672493 ] Takeshi Yamamuro commented on SPARK-18478: -- okay, I'll check > Support codegen

[jira] [Updated] (SPARK-18319) ML, Graph 2.1 QA: API: Experimental, DeveloperApi, final, sealed audit

2016-11-16 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-18319: Assignee: yuhao yang > ML, Graph 2.1 QA: API: Experimental, DeveloperApi, final, sealed audit > ---

[jira] [Updated] (SPARK-18320) ML 2.1 QA: API: Python API coverage

2016-11-16 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-18320: Assignee: Seth Hendrickson > ML 2.1 QA: API: Python API coverage >

[jira] [Commented] (SPARK-18478) Support codegen for Hive UDFs

2016-11-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15672453#comment-15672453 ] Reynold Xin commented on SPARK-18478: - Are there any performance improvements we will

[jira] [Assigned] (SPARK-18317) ML, Graph 2.1 QA: API: Binary incompatible changes

2016-11-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-18317: - Assignee: Xiangrui Meng > ML, Graph 2.1 QA: API: Binary incompatible changes > -

[jira] [Assigned] (SPARK-18449) Name option is being ignored when submitting an R application via spark-submit

2016-11-16 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reassigned SPARK-18449: Assignee: Felix Cheung > Name option is being ignored when submitting an R application via

[jira] [Commented] (SPARK-18449) Name option is being ignored when submitting an R application via spark-submit

2016-11-16 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15672403#comment-15672403 ] Felix Cheung commented on SPARK-18449: -- Good catch, this is likely because the R fun

[jira] [Commented] (SPARK-18353) spark.rpc.askTimeout defalut value is not 120s

2016-11-16 Thread Jason Pan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15672312#comment-15672312 ] Jason Pan commented on SPARK-18353: --- Thanks sean. It works. Just for the doc: "spark.

[jira] [Updated] (SPARK-18468) Flaky test: org.apache.spark.sql.hive.HiveSparkSubmitSuite.SPARK-9757 Persist Parquet relation with decimal column

2016-11-16 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-18468: - Description: https://amplab.cs.berkeley.edu/jenkins/job/spark-branch-2.1-test-sbt-hadoop-2.4/71/testRepor

[jira] [Updated] (SPARK-18468) Flaky test: org.apache.spark.sql.hive.HiveSparkSubmitSuite.SPARK-9757 Persist Parquet relation with decimal column

2016-11-16 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-18468: - Component/s: (was: SQL) Spark Core > Flaky test: org.apache.spark.sql.hive.HiveSpark

[jira] [Created] (SPARK-18479) spark.sql.shuffle.partitions defaults should be a prime number

2016-11-16 Thread Hamel Ajay Kothari (JIRA)
Hamel Ajay Kothari created SPARK-18479: -- Summary: spark.sql.shuffle.partitions defaults should be a prime number Key: SPARK-18479 URL: https://issues.apache.org/jira/browse/SPARK-18479 Project: S

[jira] [Commented] (SPARK-18478) Support codegen for Hive UDFs

2016-11-16 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15672272#comment-15672272 ] Takeshi Yamamuro commented on SPARK-18478: -- We can simply fix this (https://git

[jira] [Comment Edited] (SPARK-18473) Correctness issue in INNER join result with window functions

2016-11-16 Thread peay (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15672245#comment-15672245 ] peay edited comment on SPARK-18473 at 11/17/16 12:55 AM: - Ok, I s

[jira] [Created] (SPARK-18478) Support codegen for Hive UDFs

2016-11-16 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-18478: Summary: Support codegen for Hive UDFs Key: SPARK-18478 URL: https://issues.apache.org/jira/browse/SPARK-18478 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-18477) Enable interrupts for HDFS in HDFSMetadataLog

2016-11-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15672225#comment-15672225 ] Apache Spark commented on SPARK-18477: -- User 'zsxwing' has created a pull request fo

[jira] [Commented] (SPARK-18473) Correctness issue in INNER join result with window functions

2016-11-16 Thread peay (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15672245#comment-15672245 ] peay commented on SPARK-18473: -- Ok, I see, thanks. The fix is in 2.0.3 though, not 2.0.2, co

[jira] [Assigned] (SPARK-18477) Enable interrupts for HDFS in HDFSMetadataLog

2016-11-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18477: Assignee: Shixiong Zhu (was: Apache Spark) > Enable interrupts for HDFS in HDFSMetadataLo

[jira] [Commented] (SPARK-18473) Correctness issue in INNER join result with window functions

2016-11-16 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15672231#comment-15672231 ] Dongjoon Hyun commented on SPARK-18473: --- Hi, [~peay]. Maybe, the relevant one is ht

[jira] [Assigned] (SPARK-18477) Enable interrupts for HDFS in HDFSMetadataLog

2016-11-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18477: Assignee: Apache Spark (was: Shixiong Zhu) > Enable interrupts for HDFS in HDFSMetadataLo

[jira] [Commented] (SPARK-18473) Correctness issue in INNER join result with window functions

2016-11-16 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15672235#comment-15672235 ] Dongjoon Hyun commented on SPARK-18473: --- Wow. Please forget about my comment. :) I

[jira] [Closed] (SPARK-18473) Correctness issue in INNER join result with window functions

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-18473. - Resolution: Fixed Assignee: Xiao Li Fixed by gatorsmile's PR for SPARK-17981/SPARK-

[jira] [Commented] (SPARK-18473) Correctness issue in INNER join result with window functions

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15672223#comment-15672223 ] Herman van Hovell commented on SPARK-18473: --- This is probably caused by SPARK-1

[jira] [Commented] (SPARK-18476) SparkR Logistic Regression should should support output original label.

2016-11-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15672218#comment-15672218 ] Apache Spark commented on SPARK-18476: -- User 'wangmiao1981' has created a pull reque

[jira] [Assigned] (SPARK-18476) SparkR Logistic Regression should should support output original label.

2016-11-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18476: Assignee: Apache Spark > SparkR Logistic Regression should should support output original

[jira] [Assigned] (SPARK-18476) SparkR Logistic Regression should should support output original label.

2016-11-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18476: Assignee: (was: Apache Spark) > SparkR Logistic Regression should should support outpu

[jira] [Created] (SPARK-18477) Enable interrupts for HDFS in HDFSMetadataLog

2016-11-16 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-18477: Summary: Enable interrupts for HDFS in HDFSMetadataLog Key: SPARK-18477 URL: https://issues.apache.org/jira/browse/SPARK-18477 Project: Spark Issue Type: Imp

[jira] [Created] (SPARK-18476) SparkR Logistic Regression should should support output original label.

2016-11-16 Thread Miao Wang (JIRA)
Miao Wang created SPARK-18476: - Summary: SparkR Logistic Regression should should support output original label. Key: SPARK-18476 URL: https://issues.apache.org/jira/browse/SPARK-18476 Project: Spark

[jira] [Commented] (SPARK-18475) Be able to provide higher parallelization for StructuredStreaming Kafka Source

2016-11-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671923#comment-15671923 ] Apache Spark commented on SPARK-18475: -- User 'brkyvz' has created a pull request for

[jira] [Assigned] (SPARK-18475) Be able to provide higher parallelization for StructuredStreaming Kafka Source

2016-11-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18475: Assignee: (was: Apache Spark) > Be able to provide higher parallelization for Structur

[jira] [Assigned] (SPARK-18475) Be able to provide higher parallelization for StructuredStreaming Kafka Source

2016-11-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18475: Assignee: Apache Spark > Be able to provide higher parallelization for StructuredStreaming

[jira] [Created] (SPARK-18475) Be able to provide higher parallelization for StructuredStreaming Kafka Source

2016-11-16 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-18475: --- Summary: Be able to provide higher parallelization for StructuredStreaming Kafka Source Key: SPARK-18475 URL: https://issues.apache.org/jira/browse/SPARK-18475 Project:

[jira] [Resolved] (SPARK-18186) Migrate HiveUDAFFunction to TypedImperativeAggregate for partial aggregation support

2016-11-16 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-18186. -- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 15703 [https://github.com/

[jira] [Resolved] (SPARK-1267) Add a pip installer for PySpark

2016-11-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-1267. --- Resolution: Fixed Fix Version/s: 2.2.0 Merged into master (2.2) and will consider for 2.1. > A

[jira] [Updated] (SPARK-18129) Sign pip artifacts

2016-11-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-18129: --- Assignee: holdenk > Sign pip artifacts > -- > > Key: SPARK-18129 >

[jira] [Resolved] (SPARK-18129) Sign pip artifacts

2016-11-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-18129. Resolution: Fixed Fix Version/s: 2.2.0 Merged to master (2.2). > Sign pip artifacts > -

[jira] [Updated] (SPARK-1267) Add a pip installer for PySpark

2016-11-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-1267: -- Assignee: holdenk > Add a pip installer for PySpark > --- > >

[jira] [Updated] (SPARK-16609) Single function for parsing timestamps/dates

2016-11-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16609: Assignee: (was: Reynold Xin) > Single function for parsing timestamps/dates > -

[jira] [Resolved] (SPARK-18424) Single Function for Parsing Dates and Times with Formats

2016-11-16 Thread Bill Chambers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Chambers resolved SPARK-18424. --- Resolution: Duplicate This is a duplicate of SPARK-16609. Work will continue there. > Single

[jira] [Commented] (SPARK-16609) Single function for parsing timestamps/dates

2016-11-16 Thread Bill Chambers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671632#comment-15671632 ] Bill Chambers commented on SPARK-16609: --- I am working on this. > Single function f

[jira] [Updated] (SPARK-18424) Single Function for Parsing Dates and Times with Formats

2016-11-16 Thread Bill Chambers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Chambers updated SPARK-18424: -- Summary: Single Function for Parsing Dates and Times with Formats (was: Single Funct) > Singl

[jira] [Updated] (SPARK-18424) Single Funct

2016-11-16 Thread Bill Chambers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Chambers updated SPARK-18424: -- Summary: Single Funct (was: Improve Date Parsing Semantics & Functionality) > Single Funct > -

[jira] [Commented] (SPARK-18473) Correctness issue in INNER join result with window functions

2016-11-16 Thread peay (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671624#comment-15671624 ] peay commented on SPARK-18473: -- Ah, great, thanks. I had checked out the CHANGELOG but could

[jira] [Commented] (SPARK-18473) Correctness issue in INNER join result with window functions

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671609#comment-15671609 ] Herman van Hovell commented on SPARK-18473: --- This has been fixed in spark 2.0.2

[jira] [Comment Edited] (SPARK-18473) Correctness issue in INNER join result with window functions

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671609#comment-15671609 ] Herman van Hovell edited comment on SPARK-18473 at 11/16/16 8:59 PM: --

[jira] [Created] (SPARK-18474) Add StreamingQuery.status in python

2016-11-16 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-18474: - Summary: Add StreamingQuery.status in python Key: SPARK-18474 URL: https://issues.apache.org/jira/browse/SPARK-18474 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-18473) Correctness issue in INNER join result with window functions

2016-11-16 Thread peay (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] peay updated SPARK-18473: - Description: I have stumbled onto a corner case where an INNER join appears to return incorrect results. I belie

[jira] [Updated] (SPARK-18473) Correctness issue in INNER join result with window functions

2016-11-16 Thread peay (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] peay updated SPARK-18473: - Description: I have stumbled onto a corner case where an INNER join appears to return incorrect results. I belie

[jira] [Updated] (SPARK-18473) Correctness issue in INNER join result with window functions

2016-11-16 Thread peay (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] peay updated SPARK-18473: - Description: I have stumbled onto a corner case where an INNER join appears to return incorrect results. I belie

[jira] [Updated] (SPARK-18473) Correctness issue in INNER join result with window functions

2016-11-16 Thread peay (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] peay updated SPARK-18473: - Description: I have stumbled onto a corner case where an INNER join appears to return incorrect results. I belie

[jira] [Updated] (SPARK-18473) Correctness issue in INNER join result with window functions

2016-11-16 Thread peay (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] peay updated SPARK-18473: - Description: I have stumbled onto a corner case where an INNER join appears to return incorrect results. I belie

[jira] [Created] (SPARK-18473) Correctness issue in INNER join result with window functions

2016-11-16 Thread peay (JIRA)
peay created SPARK-18473: Summary: Correctness issue in INNER join result with window functions Key: SPARK-18473 URL: https://issues.apache.org/jira/browse/SPARK-18473 Project: Spark Issue Type: Bug

[jira] [Closed] (SPARK-16795) Spark's HiveThriftServer should be able to use multiple sqlContexts

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-16795. - Resolution: Duplicate > Spark's HiveThriftServer should be able to use multiple sqlContex

[jira] [Commented] (SPARK-16795) Spark's HiveThriftServer should be able to use multiple sqlContexts

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671462#comment-15671462 ] Herman van Hovell commented on SPARK-16795: --- Spark uses one Hive client per spa

[jira] [Resolved] (SPARK-16865) A file-based end-to-end SQL query suite

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-16865. --- Resolution: Fixed Assignee: Peter Lee Fix Version/s: 2.0.1

[jira] [Updated] (SPARK-16951) Alternative implementation of NOT IN to Anti-join

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-16951: -- Issue Type: Sub-task (was: Improvement) Parent: SPARK-18455 > Alternative impl

[jira] [Resolved] (SPARK-17268) Break Optimizer.scala apart

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17268. --- Resolution: Fixed Fix Version/s: 2.1.0 > Break Optimizer.scala apart > ---

[jira] [Commented] (SPARK-13510) Shuffle may throw FetchFailedException: Direct buffer memory

2016-11-16 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671415#comment-15671415 ] Sital Kedia commented on SPARK-13510: - [~shenhong] - We are seeing the same issue on

[jira] [Commented] (SPARK-17450) spark sql rownumber OOM

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671411#comment-15671411 ] Herman van Hovell commented on SPARK-17450: --- [~cenyuhai] did you have any luck

[jira] [Closed] (SPARK-17662) Dedup UDAF

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-17662. - Resolution: Not A Problem > Dedup UDAF > -- > > Key: SPARK-17662

[jira] [Commented] (SPARK-17662) Dedup UDAF

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671401#comment-15671401 ] Herman van Hovell commented on SPARK-17662: --- This is more of a question for the

[jira] [Commented] (SPARK-18172) AnalysisException in first/last during aggregation

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671361#comment-15671361 ] Herman van Hovell commented on SPARK-18172: --- This is different from SPARK-18300

[jira] [Resolved] (SPARK-18172) AnalysisException in first/last during aggregation

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-18172. --- Resolution: Fixed Fix Version/s: 2.0.2 Target Version/s: (was: 2.1.

[jira] [Updated] (SPARK-17786) [SPARK 2.0] Sorting algorithm gives higher skewness of output

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-17786: -- Target Version/s: 2.1.0 > [SPARK 2.0] Sorting algorithm gives higher skewness of output

[jira] [Updated] (SPARK-17788) RangePartitioner results in few very large tasks and many small to empty tasks

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-17788: -- Target Version/s: 2.1.0 > RangePartitioner results in few very large tasks and many sma

[jira] [Commented] (SPARK-17932) Failed to run SQL "show table extended like table_name" in Spark2.0.0

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671331#comment-15671331 ] Herman van Hovell commented on SPARK-17932: --- This is currently not implemented

[jira] [Updated] (SPARK-17897) not isnotnull is converted to the always false condition isnotnull && not isnotnull

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-17897: -- Target Version/s: 2.1.0 > not isnotnull is converted to the always false condition isno

[jira] [Commented] (SPARK-18460) Include triggerDetails in StreamingQueryStatus.json

2016-11-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671328#comment-15671328 ] Apache Spark commented on SPARK-18460: -- User 'tdas' has created a pull request for t

[jira] [Commented] (SPARK-18459) Rename triggerId to batchId in StreamingQueryStatus.triggerDetails

2016-11-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671327#comment-15671327 ] Apache Spark commented on SPARK-18459: -- User 'tdas' has created a pull request for t

[jira] [Comment Edited] (SPARK-17977) DataFrameReader and DataStreamReader should have an ancestor class

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671316#comment-15671316 ] Herman van Hovell edited comment on SPARK-17977 at 11/16/16 7:07 PM: --

[jira] [Commented] (SPARK-17977) DataFrameReader and DataStreamReader should have an ancestor class

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671316#comment-15671316 ] Herman van Hovell commented on SPARK-17977: --- [~aassudani] want to open a PR for

[jira] [Commented] (SPARK-18458) core dumped running Spark SQL on large data volume (100TB)

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671314#comment-15671314 ] Herman van Hovell commented on SPARK-18458: --- Nice find! > core dumped running

[jira] [Resolved] (SPARK-18461) Improve docs on StreamingQueryListener and StreamingQuery.status

2016-11-16 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-18461. -- Resolution: Fixed Issue resolved by pull request 15897 [https://github.com/apache/spark

[jira] [Commented] (SPARK-17977) DataFrameReader and DataStreamReader should have an ancestor class

2016-11-16 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671287#comment-15671287 ] Michael Armbrust commented on SPARK-17977: -- No, they were actually the same clas

[jira] [Assigned] (SPARK-18458) core dumped running Spark SQL on large data volume (100TB)

2016-11-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18458: Assignee: (was: Apache Spark) > core dumped running Spark SQL on large data volume (10

[jira] [Assigned] (SPARK-18458) core dumped running Spark SQL on large data volume (100TB)

2016-11-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18458: Assignee: Apache Spark > core dumped running Spark SQL on large data volume (100TB) >

[jira] [Commented] (SPARK-18458) core dumped running Spark SQL on large data volume (100TB)

2016-11-16 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671237#comment-15671237 ] Kazuaki Ishizaki commented on SPARK-18458: -- I worked with [~jfc...@us.ibm.com].

[jira] [Issue Comment Deleted] (SPARK-18458) core dumped running Spark SQL on large data volume (100TB)

2016-11-16 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-18458: - Comment: was deleted (was: I worked with [~jfc...@us.ibm.com]. Then, I identified that a

[jira] [Commented] (SPARK-18458) core dumped running Spark SQL on large data volume (100TB)

2016-11-16 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671229#comment-15671229 ] Kazuaki Ishizaki commented on SPARK-18458: -- I worked with [~jfc...@us.ibm.com].

[jira] [Commented] (SPARK-18172) AnalysisException in first/last during aggregation

2016-11-16 Thread Emlyn Corrin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671227#comment-15671227 ] Emlyn Corrin commented on SPARK-18172: -- It occurs on 2.0.1 and 2.0.2 (on Mac, instal

  1   2   3   >