[jira] [Commented] (SPARK-17811) SparkR cannot parallelize data.frame with NA or NULL in Date columns

2016-10-06 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554187#comment-15554187 ] Miao Wang commented on SPARK-17811: --- I am looking at this issue. Thanks! > SparkR cannot parallelize

[jira] [Updated] (SPARK-17612) Support `DESCRIBE table PARTITION` SQL syntax

2016-10-06 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-17612: -- Fix Version/s: 2.0.2 > Support `DESCRIBE table PARTITION` SQL syntax >

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-10-06 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554160#comment-15554160 ] Cody Koeninger commented on SPARK-15406: I think if you're already gravitating towards json as

[jira] [Assigned] (SPARK-17817) PySpark RDD Repartitioning Results in Highly Skewed Partition Sizes

2016-10-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17817: Assignee: Apache Spark > PySpark RDD Repartitioning Results in Highly Skewed Partition

[jira] [Commented] (SPARK-17817) PySpark RDD Repartitioning Results in Highly Skewed Partition Sizes

2016-10-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554157#comment-15554157 ] Apache Spark commented on SPARK-17817: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17817) PySpark RDD Repartitioning Results in Highly Skewed Partition Sizes

2016-10-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17817: Assignee: (was: Apache Spark) > PySpark RDD Repartitioning Results in Highly Skewed

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-10-06 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554150#comment-15554150 ] Michael Armbrust commented on SPARK-15406: -- The SQL offsets are stored in a WAL at

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-10-06 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554143#comment-15554143 ] Reynold Xin commented on SPARK-15406: - Would it make sense to create a standard format for the log so

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-10-06 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554139#comment-15554139 ] Cody Koeninger commented on SPARK-15406: When something has gone wrong, as an end user, how do I

[jira] [Updated] (SPARK-17792) L-BFGS solver for linear regression does not accept general numeric label column types

2016-10-06 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-17792: Fix Version/s: 2.0.2 > L-BFGS solver for linear regression does not accept general numeric label

[jira] [Resolved] (SPARK-17792) L-BFGS solver for linear regression does not accept general numeric label column types

2016-10-06 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang resolved SPARK-17792. - Resolution: Fixed Assignee: Seth Hendrickson Fix Version/s: 2.1.0 > L-BFGS

[jira] [Updated] (SPARK-17822) JVMObjectTracker.objMap may leak JVM objects

2016-10-06 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-17822: - Target Version/s: 1.0.2, 2.1.0 > JVMObjectTracker.objMap may leak JVM objects >

[jira] [Updated] (SPARK-17822) JVMObjectTracker.objMap may leak JVM objects

2016-10-06 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-17822: - Target Version/s: 2.0.2, 2.1.0 (was: 1.0.2, 2.1.0) > JVMObjectTracker.objMap may leak JVM objects >

[jira] [Created] (SPARK-17823) Make JVMObjectTracker.objMap thread-safe

2016-10-06 Thread Yin Huai (JIRA)
Yin Huai created SPARK-17823: Summary: Make JVMObjectTracker.objMap thread-safe Key: SPARK-17823 URL: https://issues.apache.org/jira/browse/SPARK-17823 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-10-06 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554027#comment-15554027 ] Michael Armbrust commented on SPARK-15406: -- It would be helpful if you could enumerate problems

[jira] [Created] (SPARK-17822) JVMObjectTracker.objMap may leak JVM objects

2016-10-06 Thread Yin Huai (JIRA)
Yin Huai created SPARK-17822: Summary: JVMObjectTracker.objMap may leak JVM objects Key: SPARK-17822 URL: https://issues.apache.org/jira/browse/SPARK-17822 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-17818) Cannot SELECT NULL

2016-10-06 Thread Todd Nemet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Nemet updated SPARK-17818: --- Description: When connecting to Spark SQL via JDBC/thriftserver, running `SELECT NULL` returns this

[jira] [Commented] (SPARK-17818) Cannot SELECT NULL

2016-10-06 Thread Todd Nemet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553989#comment-15553989 ] Todd Nemet commented on SPARK-17818: Oh man, you are right. I cut and pasted and wrong part of my

[jira] [Comment Edited] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-10-06 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553955#comment-15553955 ] Cody Koeninger edited comment on SPARK-15406 at 10/7/16 3:20 AM: - As soon

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-10-06 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553955#comment-15553955 ] Cody Koeninger commented on SPARK-15406: As soon as you say "checkpoint", a whole class of people

[jira] [Assigned] (SPARK-17821) Expression Canonicalization should support Add and Or

2016-10-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17821: Assignee: (was: Apache Spark) > Expression Canonicalization should support Add and Or

[jira] [Commented] (SPARK-17821) Expression Canonicalization should support Add and Or

2016-10-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553942#comment-15553942 ] Apache Spark commented on SPARK-17821: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17821) Expression Canonicalization should support Add and Or

2016-10-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17821: Assignee: Apache Spark > Expression Canonicalization should support Add and Or >

[jira] [Created] (SPARK-17821) Expression Canonicalization should support Add and Or

2016-10-06 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-17821: --- Summary: Expression Canonicalization should support Add and Or Key: SPARK-17821 URL: https://issues.apache.org/jira/browse/SPARK-17821 Project: Spark

[jira] [Commented] (SPARK-17815) Report committed offsets

2016-10-06 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553900#comment-15553900 ] Michael Armbrust commented on SPARK-17815: -- The only reason its blocked is because setting it

[jira] [Created] (SPARK-17820) Spark sqlContext.sql() performs only first insert for HiveQL "FROM target INSERT INTO dest" command to insert into multiple target tables from same source

2016-10-06 Thread Kiran Miryala (JIRA)
Kiran Miryala created SPARK-17820: - Summary: Spark sqlContext.sql() performs only first insert for HiveQL "FROM target INSERT INTO dest" command to insert into multiple target tables from same source Key: SPARK-17820

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-10-06 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553852#comment-15553852 ] Michael Armbrust commented on SPARK-15406: -- Restarting a query is just running that query again,

[jira] [Commented] (SPARK-17818) Cannot SELECT NULL

2016-10-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553798#comment-15553798 ] Dongjoon Hyun commented on SPARK-17818: --- Ur, Todd. Could you look at your JIRA description again?

[jira] [Comment Edited] (SPARK-17818) Cannot SELECT NULL

2016-10-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553798#comment-15553798 ] Dongjoon Hyun edited comment on SPARK-17818 at 10/7/16 1:37 AM: Ur, Todd.

[jira] [Commented] (SPARK-17818) Cannot SELECT NULL

2016-10-06 Thread Todd Nemet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553792#comment-15553792 ] Todd Nemet commented on SPARK-17818: I'm referring to your example above of SELECT CAST(NULL AS

[jira] [Commented] (SPARK-17818) Cannot SELECT NULL

2016-10-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553785#comment-15553785 ] Dongjoon Hyun commented on SPARK-17818: --- Thanks. > Cannot SELECT NULL > -- > >

[jira] [Commented] (SPARK-17818) Cannot SELECT NULL

2016-10-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553781#comment-15553781 ] Dongjoon Hyun commented on SPARK-17818: --- It's your example, [~tnemet]. :) > Cannot SELECT NULL >

[jira] [Closed] (SPARK-17818) Cannot SELECT NULL

2016-10-06 Thread Todd Nemet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Nemet closed SPARK-17818. -- Resolution: Duplicate Fix Version/s: 2.0.2 > Cannot SELECT NULL > -- > >

[jira] [Commented] (SPARK-17818) Cannot SELECT NULL

2016-10-06 Thread Todd Nemet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553779#comment-15553779 ] Todd Nemet commented on SPARK-17818: Definitely a duplicate. I did a search before filing - not sure

[jira] [Commented] (SPARK-17818) Cannot SELECT NULL

2016-10-06 Thread Todd Nemet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553777#comment-15553777 ] Todd Nemet commented on SPARK-17818: I don't believe you can CAST AS NULL in SQL since NullType isn't

[jira] [Created] (SPARK-17819) Specified database in JDBC URL is ignored when connecting to thriftserver

2016-10-06 Thread Todd Nemet (JIRA)
Todd Nemet created SPARK-17819: -- Summary: Specified database in JDBC URL is ignored when connecting to thriftserver Key: SPARK-17819 URL: https://issues.apache.org/jira/browse/SPARK-17819 Project: Spark

[jira] [Commented] (SPARK-17782) Kafka 010 test is flaky

2016-10-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553761#comment-15553761 ] Apache Spark commented on SPARK-17782: -- User 'koeninger' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-17818) Cannot SELECT NULL

2016-10-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553726#comment-15553726 ] Dongjoon Hyun edited comment on SPARK-17818 at 10/7/16 1:15 AM: Hi,

[jira] [Commented] (SPARK-17818) Cannot SELECT NULL

2016-10-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553745#comment-15553745 ] Dongjoon Hyun commented on SPARK-17818: --- Could you close this issue as 'Duplicate' of SPARK-17112

[jira] [Commented] (SPARK-17818) Cannot SELECT NULL

2016-10-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553738#comment-15553738 ] Dongjoon Hyun commented on SPARK-17818: --- Now, everything seems to work. BTW, the first example is

[jira] [Commented] (SPARK-17818) Cannot SELECT NULL

2016-10-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553732#comment-15553732 ] Dongjoon Hyun commented on SPARK-17818: --- {code} Connected to: Spark SQL (version 2.1.0-SNAPSHOT)

[jira] [Commented] (SPARK-17818) Cannot SELECT NULL

2016-10-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553726#comment-15553726 ] Dongjoon Hyun commented on SPARK-17818: --- Hi, [~tnemet]. It looks similar with SPARK-17112 . I'll

[jira] [Created] (SPARK-17818) Cannot SELECT NULL

2016-10-06 Thread Todd Nemet (JIRA)
Todd Nemet created SPARK-17818: -- Summary: Cannot SELECT NULL Key: SPARK-17818 URL: https://issues.apache.org/jira/browse/SPARK-17818 Project: Spark Issue Type: Bug Components: SQL

[jira] [Commented] (SPARK-17808) BinaryType fails in Python 3 due to outdated Pyrolite

2016-10-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553685#comment-15553685 ] Apache Spark commented on SPARK-17808: -- User 'BryanCutler' has created a pull request for this

[jira] [Assigned] (SPARK-17808) BinaryType fails in Python 3 due to outdated Pyrolite

2016-10-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17808: Assignee: Apache Spark > BinaryType fails in Python 3 due to outdated Pyrolite >

[jira] [Assigned] (SPARK-17808) BinaryType fails in Python 3 due to outdated Pyrolite

2016-10-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17808: Assignee: (was: Apache Spark) > BinaryType fails in Python 3 due to outdated Pyrolite

[jira] [Commented] (SPARK-17815) Report committed offsets

2016-10-06 Thread Ofir Manor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553662#comment-15553662 ] Ofir Manor commented on SPARK-17815: I think this is a good idea.There is a minor confusion here,

[jira] [Commented] (SPARK-17814) spark submit arguments are truncated in yarn-cluster mode

2016-10-06 Thread shreyas subramanya (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553633#comment-15553633 ] shreyas subramanya commented on SPARK-17814: I have tried to make the description a little

[jira] [Updated] (SPARK-17814) spark submit arguments are truncated in yarn-cluster mode

2016-10-06 Thread shreyas subramanya (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shreyas subramanya updated SPARK-17814: --- Description: {noformat} One of our customers is trying to pass in json through

[jira] [Created] (SPARK-17817) PySpark RDD Repartitioning Results in Highly Skewed Partition Sizes

2016-10-06 Thread Mike Dusenberry (JIRA)
Mike Dusenberry created SPARK-17817: --- Summary: PySpark RDD Repartitioning Results in Highly Skewed Partition Sizes Key: SPARK-17817 URL: https://issues.apache.org/jira/browse/SPARK-17817 Project:

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-10-06 Thread Ofir Manor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553613#comment-15553613 ] Ofir Manor commented on SPARK-15406: I see three somewhat-related issues: 1. More flexibility is

[jira] [Assigned] (SPARK-17816) Json serialzation of accumulators are failing with ConcurrentModificationException

2016-10-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17816: Assignee: (was: Apache Spark) > Json serialzation of accumulators are failing with >

[jira] [Commented] (SPARK-17816) Json serialzation of accumulators are failing with ConcurrentModificationException

2016-10-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553608#comment-15553608 ] Apache Spark commented on SPARK-17816: -- User 'seyfe' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17816) Json serialzation of accumulators are failing with ConcurrentModificationException

2016-10-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17816: Assignee: Apache Spark > Json serialzation of accumulators are failing with >

[jira] [Commented] (SPARK-17732) ALTER TABLE DROP PARTITION should support comparators

2016-10-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553593#comment-15553593 ] Dongjoon Hyun commented on SPARK-17732: --- My bad. I didn't search properly the existing issue for

[jira] [Commented] (SPARK-17814) spark submit arguments are truncated in yarn-cluster mode

2016-10-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553553#comment-15553553 ] Sean Owen commented on SPARK-17814: --- I think the JIRA formatting is obscuring what you're exactly

[jira] [Comment Edited] (SPARK-17814) spark submit arguments are truncated in yarn-cluster mode

2016-10-06 Thread shreyas subramanya (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553526#comment-15553526 ] shreyas subramanya edited comment on SPARK-17814 at 10/6/16 11:19 PM:

[jira] [Commented] (SPARK-17814) spark submit arguments are truncated in yarn-cluster mode

2016-10-06 Thread shreyas subramanya (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553526#comment-15553526 ] shreyas subramanya commented on SPARK-17814: well, the same command works fine in yarn-client

[jira] [Updated] (SPARK-17780) Report NoClassDefFoundError in StreamExecution

2016-10-06 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-17780: Assignee: Shixiong Zhu > Report NoClassDefFoundError in StreamExecution >

[jira] [Commented] (SPARK-12837) Spark driver requires large memory space for serialized results even there are no data collected to the driver

2016-10-06 Thread Jason Herman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553518#comment-15553518 ] Jason Herman commented on SPARK-12837: -- This bug is not fixed in spark 2.0 Here are the UPDATED

[jira] [Commented] (SPARK-17647) SQL LIKE does not handle backslashes correctly

2016-10-06 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553517#comment-15553517 ] Jakob Odersky commented on SPARK-17647: --- Xiao pointed me to this issue, I can take a look at it >

[jira] [Updated] (SPARK-17814) spark submit arguments are truncated in yarn-cluster mode

2016-10-06 Thread shreyas subramanya (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shreyas subramanya updated SPARK-17814: --- Description: One of our customers is trying to pass in json through spark-submit as

[jira] [Updated] (SPARK-17814) spark submit arguments are truncated in yarn-cluster mode

2016-10-06 Thread shreyas subramanya (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shreyas subramanya updated SPARK-17814: --- Description: One of our customers is trying to pass in json through spark-submit as

[jira] [Updated] (SPARK-17814) spark submit arguments are truncated in yarn-cluster mode

2016-10-06 Thread shreyas subramanya (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shreyas subramanya updated SPARK-17814: --- Description: One of our customers is trying to pass in json through spark-submit as

[jira] [Updated] (SPARK-17750) Cannot create view which includes interval arithmetic

2016-10-06 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-17750: Fix Version/s: 2.0.2 > Cannot create view which includes interval arithmetic >

[jira] [Commented] (SPARK-17219) QuantileDiscretizer does strange things with NaN values

2016-10-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553503#comment-15553503 ] Sean Owen commented on SPARK-17219: --- I could go this way too. I ended up sympathizing with trying to

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-10-06 Thread Yuval Itzchakov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553493#comment-15553493 ] Yuval Itzchakov commented on SPARK-15406: - We'ee using Kafka in a combination with stateful

[jira] [Commented] (SPARK-17219) QuantileDiscretizer does strange things with NaN values

2016-10-06 Thread Timothy Hunter (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553490#comment-15553490 ] Timothy Hunter commented on SPARK-17219: If I understand correctly the PR, I am concerned by this

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-10-06 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553476#comment-15553476 ] Cody Koeninger commented on SPARK-15406: You cannot have reliable delivery semantics to a

[jira] [Updated] (SPARK-17816) Json serialzation of accumulators are failing with ConcurrentModificationException

2016-10-06 Thread Ergin Seyfe (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ergin Seyfe updated SPARK-17816: Description: This is the stack trace: See {{ConcurrentModificationException}}: {code}

[jira] [Created] (SPARK-17816) Json serialzation of accumulators are failing with ConcurrentModificationException

2016-10-06 Thread Ergin Seyfe (JIRA)
Ergin Seyfe created SPARK-17816: --- Summary: Json serialzation of accumulators are failing with ConcurrentModificationException Key: SPARK-17816 URL: https://issues.apache.org/jira/browse/SPARK-17816

[jira] [Commented] (SPARK-17814) spark submit arguments are truncated in yarn-cluster mode

2016-10-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553434#comment-15553434 ] Sean Owen commented on SPARK-17814: --- It looks like you're not escaping these reserved chars in your

[jira] [Created] (SPARK-17815) Report committed offsets

2016-10-06 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-17815: Summary: Report committed offsets Key: SPARK-17815 URL: https://issues.apache.org/jira/browse/SPARK-17815 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-17814) spark submit arguments are truncated in yarn-cluster mode

2016-10-06 Thread shreyas subramanya (JIRA)
shreyas subramanya created SPARK-17814: -- Summary: spark submit arguments are truncated in yarn-cluster mode Key: SPARK-17814 URL: https://issues.apache.org/jira/browse/SPARK-17814 Project: Spark

[jira] [Commented] (SPARK-17647) SQL LIKE does not handle backslashes correctly

2016-10-06 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553424#comment-15553424 ] Xiao Li commented on SPARK-17647: - Sure, let me check it with my teammates. Thanks! > SQL LIKE does not

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-10-06 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553405#comment-15553405 ] Michael Armbrust commented on SPARK-15406: -- The programming guide has a pretty good description

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-10-06 Thread Yuval Itzchakov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553391#comment-15553391 ] Yuval Itzchakov commented on SPARK-15406: - @Michael Perhaps I need to read that PR through to

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-10-06 Thread Ofir Manor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553372#comment-15553372 ] Ofir Manor commented on SPARK-15406: Thanks Michael, I'll check the JIRAs, and will also post some

[jira] [Commented] (SPARK-17647) SQL LIKE does not handle backslashes correctly

2016-10-06 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553357#comment-15553357 ] Yin Huai commented on SPARK-17647: -- [~smilegator] Anyone from your side has time to take a look at it?

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-10-06 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553328#comment-15553328 ] Michael Armbrust commented on SPARK-15406: -- This missed 2.0.1, but I'm hopeful it will be in

[jira] [Commented] (SPARK-17346) Kafka 0.10 support in Structured Streaming

2016-10-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553316#comment-15553316 ] Apache Spark commented on SPARK-17346: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-10-06 Thread Yuval Itzchakov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553301#comment-15553301 ] Yuval Itzchakov commented on SPARK-15406: - As someone using Spark Streaming and Kafka in

[jira] [Updated] (SPARK-17803) Docker integration tests don't run with "Docker for Mac"

2016-10-06 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17803: --- Assignee: Christian Kadner > Docker integration tests don't run with "Docker for Mac" >

[jira] [Resolved] (SPARK-17803) Docker integration tests don't run with "Docker for Mac"

2016-10-06 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-17803. Resolution: Fixed Fix Version/s: 2.1.0 2.0.2 Issue resolved by pull

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-10-06 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553212#comment-15553212 ] Michael Armbrust commented on SPARK-15406: -- We are definitly not done! I added a few extra

[jira] [Created] (SPARK-17813) Maximum data per trigger

2016-10-06 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-17813: Summary: Maximum data per trigger Key: SPARK-17813 URL: https://issues.apache.org/jira/browse/SPARK-17813 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-10-06 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553176#comment-15553176 ] Cody Koeninger commented on SPARK-15406: I'm talking about people using Spark, not committers.

[jira] [Created] (SPARK-17812) More granular control of starting offsets

2016-10-06 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-17812: Summary: More granular control of starting offsets Key: SPARK-17812 URL: https://issues.apache.org/jira/browse/SPARK-17812 Project: Spark Issue

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-10-06 Thread Ofir Manor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553161#comment-15553161 ] Ofir Manor commented on SPARK-15406: Alright, I do hope you all find a way to resolve this and switch

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-10-06 Thread Jacek Laskowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553158#comment-15553158 ] Jacek Laskowski commented on SPARK-15406: - I'm curious who the users were? You're talking about

[jira] [Resolved] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-10-06 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17463. -- Resolution: Fixed It's already fixed. The issue in the comment is another one. >

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-10-06 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553134#comment-15553134 ] Shixiong Zhu commented on SPARK-17463: -- [~eseyfe] It's a different issue. Could you create a new

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-10-06 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553099#comment-15553099 ] Cody Koeninger commented on SPARK-15406: To be real blunt, my level of interest is pretty low in

[jira] [Commented] (SPARK-17131) Code generation fails when running SQL expressions against a wide dataset (thousands of columns)

2016-10-06 Thread Andrey Melentyev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553101#comment-15553101 ] Andrey Melentyev commented on SPARK-17131: -- Looks similar to

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-10-06 Thread Ofir Manor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553076#comment-15553076 ] Ofir Manor commented on SPARK-15406: Thanks Cody! I missed the activity in the subtask PR... It is

[jira] [Created] (SPARK-17811) SparkR cannot parallelize data.frame with NA or NULL in Date columns

2016-10-06 Thread Hossein Falaki (JIRA)
Hossein Falaki created SPARK-17811: -- Summary: SparkR cannot parallelize data.frame with NA or NULL in Date columns Key: SPARK-17811 URL: https://issues.apache.org/jira/browse/SPARK-17811 Project:

[jira] [Commented] (SPARK-17750) Cannot create view which includes interval arithmetic

2016-10-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553004#comment-15553004 ] Apache Spark commented on SPARK-17750: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Resolved] (SPARK-17780) Report NoClassDefFoundError in StreamExecution

2016-10-06 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-17780. -- Resolution: Fixed Fix Version/s: 2.1.0 2.0.2 Issue resolved

[jira] [Commented] (SPARK-17131) Code generation fails when running SQL expressions against a wide dataset (thousands of columns)

2016-10-06 Thread Andrey Melentyev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15552994#comment-15552994 ] Andrey Melentyev commented on SPARK-17131: -- I tried wrapping the attached test into

[jira] [Assigned] (SPARK-17810) Default spark.sql.warehouse.dir is relative to local FS but can resolve as HDFS path

2016-10-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17810: Assignee: Apache Spark (was: Sean Owen) > Default spark.sql.warehouse.dir is relative to

  1   2   >