[jira] [Commented] (SPARK-16950) fromOffsets parameter in Kafka's Direct Streams does not work in python3

2016-10-05 Thread Mariusz Strzelecki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550948#comment-15550948 ] Mariusz Strzelecki commented on SPARK-16950: Hello Russel, your code works in my environment.

[jira] [Comment Edited] (SPARK-16950) fromOffsets parameter in Kafka's Direct Streams does not work in python3

2016-10-05 Thread Mariusz Strzelecki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550948#comment-15550948 ] Mariusz Strzelecki edited comment on SPARK-16950 at 10/6/16 5:38 AM: -

[jira] [Assigned] (SPARK-17800) Introduce InterfaceStability annotation definition

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17800: Assignee: Reynold Xin (was: Apache Spark) > Introduce InterfaceStability annotation

[jira] [Assigned] (SPARK-17800) Introduce InterfaceStability annotation definition

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17800: Assignee: Apache Spark (was: Reynold Xin) > Introduce InterfaceStability annotation

[jira] [Commented] (SPARK-17800) Introduce InterfaceStability annotation definition

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550857#comment-15550857 ] Apache Spark commented on SPARK-17800: -- User 'rxin' has created a pull request for this issue:

[jira] [Created] (SPARK-17800) Introduce InterfaceStability annotation definition

2016-10-05 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-17800: --- Summary: Introduce InterfaceStability annotation definition Key: SPARK-17800 URL: https://issues.apache.org/jira/browse/SPARK-17800 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-17798) Remove redundant Experimental annotations in sql.streaming package

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17798: Assignee: Apache Spark (was: Reynold Xin) > Remove redundant Experimental annotations in

[jira] [Commented] (SPARK-17798) Remove redundant Experimental annotations in sql.streaming package

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550840#comment-15550840 ] Apache Spark commented on SPARK-17798: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17798) Remove redundant Experimental annotations in sql.streaming package

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17798: Assignee: Reynold Xin (was: Apache Spark) > Remove redundant Experimental annotations in

[jira] [Created] (SPARK-17798) Remove redundant Experimental annotations in sql.streaming package

2016-10-05 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-17798: --- Summary: Remove redundant Experimental annotations in sql.streaming package Key: SPARK-17798 URL: https://issues.apache.org/jira/browse/SPARK-17798 Project: Spark

[jira] [Commented] (SPARK-17626) TPC-DS performance improvements using star-schema heuristics

2016-10-05 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550822#comment-15550822 ] Xiao Li commented on SPARK-17626: - The short answer is provided in the design doc: {noformat} The new

[jira] [Comment Edited] (SPARK-17626) TPC-DS performance improvements using star-schema heuristics

2016-10-05 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550822#comment-15550822 ] Xiao Li edited comment on SPARK-17626 at 10/6/16 4:17 AM: -- The short answer is

[jira] [Commented] (SPARK-17626) TPC-DS performance improvements using star-schema heuristics

2016-10-05 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550808#comment-15550808 ] Reynold Xin commented on SPARK-17626: - Can you guys comment on the usefulness of this in the context

[jira] [Comment Edited] (SPARK-17626) TPC-DS performance improvements using star-schema heuristics

2016-10-05 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550799#comment-15550799 ] Xiao Li edited comment on SPARK-17626 at 10/6/16 4:02 AM: -- Selectivity Hint is

[jira] [Commented] (SPARK-17626) TPC-DS performance improvements using star-schema heuristics

2016-10-05 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550799#comment-15550799 ] Xiao Li commented on SPARK-17626: - Selectivity Hint is another one. The current JIRA `Join reordering

[jira] [Assigned] (SPARK-17797) LabelCol support non-double datatypes

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17797: Assignee: Apache Spark > LabelCol support non-double datatypes >

[jira] [Assigned] (SPARK-17797) LabelCol support non-double datatypes

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17797: Assignee: (was: Apache Spark) > LabelCol support non-double datatypes >

[jira] [Commented] (SPARK-17797) LabelCol support non-double datatypes

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550793#comment-15550793 ] Apache Spark commented on SPARK-17797: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-17797) LabelCol support non-double datatypes

2016-10-05 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550768#comment-15550768 ] zhengruifeng commented on SPARK-17797: -- cc [~sethah] > LabelCol support non-double datatypes >

[jira] [Commented] (SPARK-17626) TPC-DS performance improvements using star-schema heuristics

2016-10-05 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550764#comment-15550764 ] Reynold Xin commented on SPARK-17626: - Is there more than one task to be done here? > TPC-DS

[jira] [Created] (SPARK-17797) LabelCol support non-double datatypes

2016-10-05 Thread zhengruifeng (JIRA)
zhengruifeng created SPARK-17797: Summary: LabelCol support non-double datatypes Key: SPARK-17797 URL: https://issues.apache.org/jira/browse/SPARK-17797 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-17796) spark HiveThriftServer2 sql AnalysisException: LOAD DATA input path does not exist. if sql query is existed wild card characters

2016-10-05 Thread Tran Quyet Thang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550626#comment-15550626 ] Tran Quyet Thang commented on SPARK-17796: -- Yes, I also tried with Spark SQL - Spark Shell. >

[jira] [Commented] (SPARK-17796) spark HiveThriftServer2 sql AnalysisException: LOAD DATA input path does not exist. if sql query is existed wild card characters

2016-10-05 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550619#comment-15550619 ] Dongjoon Hyun commented on SPARK-17796: --- It's a Spark SQL issue. *Spark 2.0* {code} scala>

[jira] [Commented] (SPARK-17796) spark HiveThriftServer2 sql AnalysisException: LOAD DATA input path does not exist. if sql query is existed wild card characters

2016-10-05 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550613#comment-15550613 ] Dongjoon Hyun commented on SPARK-17796: --- I got the same result. Spark 1.6.2 works, and current

[jira] [Commented] (SPARK-17789) Don't force users to set k for KMeans if initial model is set

2016-10-05 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550607#comment-15550607 ] DB Tsai commented on SPARK-17789: - I have comment on the code. When the initial model is set, the k

[jira] [Commented] (SPARK-17789) Don't force users to set k for KMeans if initial model is set

2016-10-05 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550601#comment-15550601 ] Seth Hendrickson commented on SPARK-17789: -- When the model is fit, the initial model may have

[jira] [Updated] (SPARK-17789) Don't force users to set k for KMeans if initial model is set

2016-10-05 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hendrickson updated SPARK-17789: - Description: In the initial implementation of initalModel, we allow users to set the

[jira] [Commented] (SPARK-17789) Don't force users to set k for KMeans if initial model is set

2016-10-05 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550573#comment-15550573 ] DB Tsai commented on SPARK-17789: - Why the above code will throw a runtime exception? (you don't reset

[jira] [Commented] (SPARK-16950) fromOffsets parameter in Kafka's Direct Streams does not work in python3

2016-10-05 Thread Russell Jurney (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550562#comment-15550562 ] Russell Jurney commented on SPARK-16950: Probably doing something wrong, but I'm getting an

[jira] [Assigned] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17463: Assignee: Shixiong Zhu (was: Apache Spark) > Serialization of accumulators in heartbeats

[jira] [Assigned] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17463: Assignee: Apache Spark (was: Shixiong Zhu) > Serialization of accumulators in heartbeats

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550561#comment-15550561 ] Apache Spark commented on SPARK-17463: -- User 'seyfe' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17417) Fix # of partitions for RDD while checkpointing - Currently limited by 10000(%05d)

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17417: Assignee: Apache Spark > Fix # of partitions for RDD while checkpointing - Currently

[jira] [Reopened] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-10-05 Thread Ergin Seyfe (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ergin Seyfe reopened SPARK-17463: - Reopening since it still throw java.util.ConcurrentModificationException > Serialization of

[jira] [Commented] (SPARK-17417) Fix # of partitions for RDD while checkpointing - Currently limited by 10000(%05d)

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550559#comment-15550559 ] Apache Spark commented on SPARK-17417: -- User 'dhruve' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17417) Fix # of partitions for RDD while checkpointing - Currently limited by 10000(%05d)

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17417: Assignee: (was: Apache Spark) > Fix # of partitions for RDD while checkpointing -

[jira] [Commented] (SPARK-17796) spark HiveThriftServer2 sql AnalysisException: LOAD DATA input path does not exist. if sql query is existed wild card characters

2016-10-05 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550548#comment-15550548 ] Dongjoon Hyun commented on SPARK-17796: --- Thank you. I see. > spark HiveThriftServer2 sql

[jira] [Commented] (SPARK-17796) spark HiveThriftServer2 sql AnalysisException: LOAD DATA input path does not exist. if sql query is existed wild card characters

2016-10-05 Thread Tran Quyet Thang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550544#comment-15550544 ] Tran Quyet Thang commented on SPARK-17796: -- Hi Dongjoon Hyun, I removed the fix version. I

[jira] [Updated] (SPARK-17796) spark HiveThriftServer2 sql AnalysisException: LOAD DATA input path does not exist. if sql query is existed wild card characters

2016-10-05 Thread Tran Quyet Thang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tran Quyet Thang updated SPARK-17796: - Fix Version/s: (was: 1.6.2) > spark HiveThriftServer2 sql AnalysisException: LOAD

[jira] [Commented] (SPARK-17796) spark HiveThriftServer2 sql AnalysisException: LOAD DATA input path does not exist. if sql query is existed wild card characters

2016-10-05 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550536#comment-15550536 ] Dongjoon Hyun commented on SPARK-17796: --- Hi, [~thangtq]. Please remove the fix version when you

[jira] [Updated] (SPARK-17796) spark HiveThriftServer2 sql AnalysisException: LOAD DATA input path does not exist. if sql query is existed wild card characters

2016-10-05 Thread Tran Quyet Thang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tran Quyet Thang updated SPARK-17796: - Description: 1. I'm using a ETL tool and connecting to Spark2-HiveThriftServer2 over

[jira] [Updated] (SPARK-17796) spark HiveThriftServer2 sql AnalysisException: LOAD DATA input path does not exist. if sql query is existed wild card characters

2016-10-05 Thread Tran Quyet Thang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tran Quyet Thang updated SPARK-17796: - Description: 1. I'm using a ETL tool and connecting to Spark2-HiveThriftServer2 over

[jira] [Commented] (SPARK-17774) Add support for head on DataFrame Column

2016-10-05 Thread Hossein Falaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550488#comment-15550488 ] Hossein Falaki commented on SPARK-17774: I agree. I think if we decouple {{head}} from

[jira] [Created] (SPARK-17796) spark HiveThriftServer2 sql AnalysisException: LOAD DATA input path does not exist. if sql query is existed wild card characters

2016-10-05 Thread Tran Quyet Thang (JIRA)
Tran Quyet Thang created SPARK-17796: Summary: spark HiveThriftServer2 sql AnalysisException: LOAD DATA input path does not exist. if sql query is existed wild card characters Key: SPARK-17796 URL:

[jira] [Assigned] (SPARK-17795) Sorting on stage or job tables doesn’t reload page on that table

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17795: Assignee: Apache Spark > Sorting on stage or job tables doesn’t reload page on that table

[jira] [Assigned] (SPARK-17795) Sorting on stage or job tables doesn’t reload page on that table

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17795: Assignee: (was: Apache Spark) > Sorting on stage or job tables doesn’t reload page on

[jira] [Commented] (SPARK-17795) Sorting on stage or job tables doesn’t reload page on that table

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550455#comment-15550455 ] Apache Spark commented on SPARK-17795: -- User 'ajbozarth' has created a pull request for this issue:

[jira] [Commented] (SPARK-17790) Support for parallelizing R data.frame larger than 2GB

2016-10-05 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550421#comment-15550421 ] Felix Cheung commented on SPARK-17790: -- more discussion on

[jira] [Commented] (SPARK-17346) Kafka 0.10 support in Structured Streaming

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550423#comment-15550423 ] Apache Spark commented on SPARK-17346: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-17790) Support for parallelizing R data.frame larger than 2GB

2016-10-05 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550414#comment-15550414 ] Felix Cheung edited comment on SPARK-17790 at 10/6/16 12:34 AM: Yes.

[jira] [Commented] (SPARK-17795) Sorting on stage or job tables doesn’t reload page on that table

2016-10-05 Thread Alex Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550417#comment-15550417 ] Alex Bozarth commented on SPARK-17795: -- I have a fix for this and will submit a pr soon > Sorting

[jira] [Created] (SPARK-17795) Sorting on stage or job tables doesn’t reload page on that table

2016-10-05 Thread Alex Bozarth (JIRA)
Alex Bozarth created SPARK-17795: Summary: Sorting on stage or job tables doesn’t reload page on that table Key: SPARK-17795 URL: https://issues.apache.org/jira/browse/SPARK-17795 Project: Spark

[jira] [Commented] (SPARK-17790) Support for parallelizing R data.frame larger than 2GB

2016-10-05 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550414#comment-15550414 ] Felix Cheung commented on SPARK-17790: -- Yes. > Support for parallelizing R data.frame larger than

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-10-05 Thread Ergin Seyfe (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550398#comment-15550398 ] Ergin Seyfe commented on SPARK-17463: - Hi [~zsxwing], I think this bug is not fully fixed. I still

[jira] [Commented] (SPARK-17774) Add support for head on DataFrame Column

2016-10-05 Thread Oscar D. Lara Yejas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550390#comment-15550390 ] Oscar D. Lara Yejas commented on SPARK-17774: - To implement method head() only I'll be happy

[jira] [Commented] (SPARK-17346) Kafka 0.10 support in Structured Streaming

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550361#comment-15550361 ] Apache Spark commented on SPARK-17346: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Commented] (SPARK-17794) 2.0.1 not in maven central repo?

2016-10-05 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550352#comment-15550352 ] Dongjoon Hyun commented on SPARK-17794: --- It's on the way to propagate. > 2.0.1 not in maven

[jira] [Comment Edited] (SPARK-17794) 2.0.1 not in maven central repo?

2016-10-05 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550352#comment-15550352 ] Dongjoon Hyun edited comment on SPARK-17794 at 10/5/16 11:58 PM: - It's on

[jira] [Commented] (SPARK-17794) 2.0.1 not in maven central repo?

2016-10-05 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550346#comment-15550346 ] Dongjoon Hyun commented on SPARK-17794: --- Hi, [~hkiang01] it was published yesterday according to

[jira] [Updated] (SPARK-17794) 2.0.1 not in maven central repo?

2016-10-05 Thread Harrison Kiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harrison Kiang updated SPARK-17794: --- Description: Using IntelliJ IDEA 2016.2.4 Ultimate, Ubuntu 16.04 LTS with latest updates

[jira] [Updated] (SPARK-17643) Remove comparable requirement from Offset

2016-10-05 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17643: - Fix Version/s: 2.0.2 > Remove comparable requirement from Offset >

[jira] [Updated] (SPARK-17794) 2.0.1 not in maven central repo?

2016-10-05 Thread Harrison Kiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harrison Kiang updated SPARK-17794: --- Attachment: screenshot-1.png > 2.0.1 not in maven central repo? >

[jira] [Resolved] (SPARK-17346) Kafka 0.10 support in Structured Streaming

2016-10-05 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-17346. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 15102

[jira] [Updated] (SPARK-17794) 2.0.1 not in maven central repo?

2016-10-05 Thread Harrison Kiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harrison Kiang updated SPARK-17794: --- Description: Using IntelliJ IDEA 2016.2.4 Ultimate, Ubuntu 16.04 LTS with latest updates

[jira] [Commented] (SPARK-17793) Sorting on the description on the Job or Stage page doesn’t always work

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550321#comment-15550321 ] Apache Spark commented on SPARK-17793: -- User 'ajbozarth' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17793) Sorting on the description on the Job or Stage page doesn’t always work

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17793: Assignee: (was: Apache Spark) > Sorting on the description on the Job or Stage page

[jira] [Assigned] (SPARK-17793) Sorting on the description on the Job or Stage page doesn’t always work

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17793: Assignee: Apache Spark > Sorting on the description on the Job or Stage page doesn’t

[jira] [Updated] (SPARK-17794) 2.0.1 not in maven central repo?

2016-10-05 Thread Harrison Kiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harrison Kiang updated SPARK-17794: --- Summary: 2.0.1 not in maven central repo? (was: 2.0.1 not in maven central repo) > 2.0.1

[jira] [Commented] (SPARK-17157) Add multiclass logistic regression SparkR Wrapper

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550314#comment-15550314 ] Apache Spark commented on SPARK-17157: -- User 'wangmiao1981' has created a pull request for this

[jira] [Updated] (SPARK-17794) 2.0.1 not in maven central repo

2016-10-05 Thread Harrison Kiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harrison Kiang updated SPARK-17794: --- Description: Using IntelliJ IDEA 2016.2.4 Ultimate, Ubuntu 16.04 LTS with latest updates

[jira] [Created] (SPARK-17794) 2.0.1 not in maven central repo

2016-10-05 Thread Harrison Kiang (JIRA)
Harrison Kiang created SPARK-17794: -- Summary: 2.0.1 not in maven central repo Key: SPARK-17794 URL: https://issues.apache.org/jira/browse/SPARK-17794 Project: Spark Issue Type: Question

[jira] [Commented] (SPARK-17793) Sorting on the description on the Job or Stage page doesn’t always work

2016-10-05 Thread Alex Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550279#comment-15550279 ] Alex Bozarth commented on SPARK-17793: -- I have a fix for this and will be submitting the pr soon >

[jira] [Created] (SPARK-17793) Sorting on the description on the Job or Stage page doesn’t always work

2016-10-05 Thread Alex Bozarth (JIRA)
Alex Bozarth created SPARK-17793: Summary: Sorting on the description on the Job or Stage page doesn’t always work Key: SPARK-17793 URL: https://issues.apache.org/jira/browse/SPARK-17793 Project:

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

2016-10-05 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550262#comment-15550262 ] Cody Koeninger commented on SPARK-15406: See the PR in the linked subtask

[jira] [Assigned] (SPARK-17792) L-BFGS solver for linear regression does not accept general numeric label column types

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17792: Assignee: Apache Spark > L-BFGS solver for linear regression does not accept general

[jira] [Assigned] (SPARK-17792) L-BFGS solver for linear regression does not accept general numeric label column types

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17792: Assignee: (was: Apache Spark) > L-BFGS solver for linear regression does not accept

[jira] [Commented] (SPARK-17792) L-BFGS solver for linear regression does not accept general numeric label column types

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550260#comment-15550260 ] Apache Spark commented on SPARK-17792: -- User 'sethah' has created a pull request for this issue:

[jira] [Created] (SPARK-17792) L-BFGS solver for linear regression does not accept general numeric label column types

2016-10-05 Thread Seth Hendrickson (JIRA)
Seth Hendrickson created SPARK-17792: Summary: L-BFGS solver for linear regression does not accept general numeric label column types Key: SPARK-17792 URL: https://issues.apache.org/jira/browse/SPARK-17792

[jira] [Resolved] (SPARK-17758) Spark Aggregate function LAST returns null on an empty partition

2016-10-05 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-17758. -- Resolution: Fixed Fix Version/s: 2.1.0 2.0.2 Issue resolved by pull request

[jira] [Commented] (SPARK-17792) L-BFGS solver for linear regression does not accept general numeric label column types

2016-10-05 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550249#comment-15550249 ] Seth Hendrickson commented on SPARK-17792: -- I'll have a PR shortly. > L-BFGS solver for linear

[jira] [Updated] (SPARK-17758) Spark Aggregate function LAST returns null on an empty partition

2016-10-05 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-17758: - Assignee: Herman van Hovell > Spark Aggregate function LAST returns null on an empty partition >

[jira] [Commented] (SPARK-17074) generate histogram information for column

2016-10-05 Thread Zhenhua Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550228#comment-15550228 ] Zhenhua Wang commented on SPARK-17074: -- OK, I'll try to extend QuantileSummaries to get ndv's. >

[jira] [Commented] (SPARK-17791) Join reordering using star schema detection

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550178#comment-15550178 ] Apache Spark commented on SPARK-17791: -- User 'ioana-delaney' has created a pull request for this

[jira] [Assigned] (SPARK-17791) Join reordering using star schema detection

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17791: Assignee: Apache Spark > Join reordering using star schema detection >

[jira] [Assigned] (SPARK-17791) Join reordering using star schema detection

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17791: Assignee: (was: Apache Spark) > Join reordering using star schema detection >

[jira] [Updated] (SPARK-17791) Join reordering using star schema detection

2016-10-05 Thread Ioana Delaney (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ioana Delaney updated SPARK-17791: -- Attachment: StarJoinReordering1005.doc > Join reordering using star schema detection >

[jira] [Created] (SPARK-17791) Join reordering using star schema detection

2016-10-05 Thread Ioana Delaney (JIRA)
Ioana Delaney created SPARK-17791: - Summary: Join reordering using star schema detection Key: SPARK-17791 URL: https://issues.apache.org/jira/browse/SPARK-17791 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-17778) Mock SparkContext to reduce memory usage of BlockManagerSuite

2016-10-05 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17778. -- Resolution: Fixed Fix Version/s: 2.1.0 2.0.2 > Mock SparkContext to

[jira] [Commented] (SPARK-17643) Remove comparable requirement from Offset

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550048#comment-15550048 ] Apache Spark commented on SPARK-17643: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Commented] (SPARK-15369) Investigate selectively using Jython for parts of PySpark

2016-10-05 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550012#comment-15550012 ] holdenk commented on SPARK-15369: - Certainly we can investigate speeding up the serialization between the

[jira] [Updated] (SPARK-17790) Support for parallelizing data.frame larger than 2GB

2016-10-05 Thread Hossein Falaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hossein Falaki updated SPARK-17790: --- Issue Type: Sub-task (was: Story) Parent: SPARK-6235 > Support for parallelizing

[jira] [Updated] (SPARK-17790) Support for parallelizing R data.frame larger than 2GB

2016-10-05 Thread Hossein Falaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hossein Falaki updated SPARK-17790: --- Summary: Support for parallelizing R data.frame larger than 2GB (was: Support for

[jira] [Commented] (SPARK-17790) Support for parallelizing data.frame larger than 2GB

2016-10-05 Thread Hossein Falaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549996#comment-15549996 ] Hossein Falaki commented on SPARK-17790: Thanks for pointing it out. SPARK-6235 seems to be an

[jira] [Commented] (SPARK-15369) Investigate selectively using Jython for parts of PySpark

2016-10-05 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549990#comment-15549990 ] Reynold Xin commented on SPARK-15369: - So while I'm sure you can improve performance for some UDFs,

[jira] [Commented] (SPARK-17790) Support for parallelizing data.frame larger than 2GB

2016-10-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549980#comment-15549980 ] Sean Owen commented on SPARK-17790: --- This duplicates https://issues.apache.org/jira/browse/SPARK-6235 ?

[jira] [Updated] (SPARK-17790) Support for parallelizing data.frame larger than 2GB

2016-10-05 Thread Hossein Falaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hossein Falaki updated SPARK-17790: --- Summary: Support for parallelizing data.frame larger than 2GB (was: Support for

[jira] [Commented] (SPARK-17790) Support for parallelizing/creating DataFrame on data larger than 2GB

2016-10-05 Thread Hossein Falaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549932#comment-15549932 ] Hossein Falaki commented on SPARK-17790: [~shivaram] and [~mengxr] just double checking that in

[jira] [Created] (SPARK-17790) Support for parallelizing/creating DataFrame on data larger than 2GB

2016-10-05 Thread Hossein Falaki (JIRA)
Hossein Falaki created SPARK-17790: -- Summary: Support for parallelizing/creating DataFrame on data larger than 2GB Key: SPARK-17790 URL: https://issues.apache.org/jira/browse/SPARK-17790 Project:

[jira] [Commented] (SPARK-17774) Add support for head on DataFrame Column

2016-10-05 Thread Hossein Falaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549913#comment-15549913 ] Hossein Falaki commented on SPARK-17774: I strongly feel {{head}} should work, but I don't have

[jira] [Created] (SPARK-17789) Don't force users to set k for KMeans if initial model is set

2016-10-05 Thread Seth Hendrickson (JIRA)
Seth Hendrickson created SPARK-17789: Summary: Don't force users to set k for KMeans if initial model is set Key: SPARK-17789 URL: https://issues.apache.org/jira/browse/SPARK-17789 Project: Spark

  1   2   >