[jira] [Assigned] (SPARK-20590) Map default input data source formats to inlined classes

2017-05-09 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20590: --- Assignee: Hyukjin Kwon > Map default input data source formats to inlined classes >

[jira] [Resolved] (SPARK-20590) Map default input data source formats to inlined classes

2017-05-09 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20590. - Resolution: Fixed Fix Version/s: 2.3.0 2.2.1 Issue resolved by pull

[jira] [Commented] (SPARK-12837) Spark driver requires large memory space for serialized results even there are no data collected to the driver

2017-05-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004103#comment-16004103 ] Apache Spark commented on SPARK-12837: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20688) correctly check analysis for scalar sub-queries

2017-05-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20688: Assignee: Apache Spark (was: Wenchen Fan) > correctly check analysis for scalar

[jira] [Assigned] (SPARK-20688) correctly check analysis for scalar sub-queries

2017-05-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20688: Assignee: Wenchen Fan (was: Apache Spark) > correctly check analysis for scalar

[jira] [Commented] (SPARK-20688) correctly check analysis for scalar sub-queries

2017-05-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004060#comment-16004060 ] Apache Spark commented on SPARK-20688: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Created] (SPARK-20688) correctly check analysis for scalar sub-queries

2017-05-09 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-20688: --- Summary: correctly check analysis for scalar sub-queries Key: SPARK-20688 URL: https://issues.apache.org/jira/browse/SPARK-20688 Project: Spark Issue Type:

[jira] [Issue Comment Deleted] (SPARK-20687) mllib.Matrices.fromBreeze may crash when converting breeze CSCMatrix

2017-05-09 Thread Ignacio Bermudez Corrales (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ignacio Bermudez Corrales updated SPARK-20687: -- Comment: was deleted (was: test("breeze conversion bug") { //

[jira] [Created] (SPARK-20687) mllib.Matrices.fromBreeze may crash when converting breeze CSCMatrix

2017-05-09 Thread Ignacio Bermudez Corrales (JIRA)
Ignacio Bermudez Corrales created SPARK-20687: - Summary: mllib.Matrices.fromBreeze may crash when converting breeze CSCMatrix Key: SPARK-20687 URL: https://issues.apache.org/jira/browse/SPARK-20687

[jira] [Commented] (SPARK-20554) Remove usage of scala.language.reflectiveCalls

2017-05-09 Thread Umesh Chaudhary (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004020#comment-16004020 ] Umesh Chaudhary commented on SPARK-20554: - [~srowen] Recently I did not get time to through all

[jira] [Comment Edited] (SPARK-14584) Improve recognition of non-nullability in Dataset transformations

2017-05-09 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003996#comment-16003996 ] Hyukjin Kwon edited comment on SPARK-14584 at 5/10/17 3:56 AM: ---

[jira] [Commented] (SPARK-14584) Improve recognition of non-nullability in Dataset transformations

2017-05-09 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003996#comment-16003996 ] Hyukjin Kwon commented on SPARK-14584: -- [~joshrosen] SPARK-18284 fixes this. Before this PR:

[jira] [Updated] (SPARK-20665) Spark-sql, "Bround" and "Round" function return NULL

2017-05-09 Thread liuxian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuxian updated SPARK-20665: Summary: Spark-sql, "Bround" and "Round" function return NULL (was: Spark-sql, "Bround" function return

[jira] [Resolved] (SPARK-17685) WholeStageCodegenExec throws IndexOutOfBoundsException

2017-05-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17685. --- Resolution: Fixed Assignee: Yuming Wang Fix Version/s: 2.3.0

[jira] [Commented] (SPARK-18528) limit + groupBy leads to java.lang.NullPointerException

2017-05-09 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003882#comment-16003882 ] Takeshi Yamamuro commented on SPARK-18528: -- You tried spark-v2.1.1? This issue is fixed in

[jira] [Commented] (SPARK-20683) Make table uncache chaining optional

2017-05-09 Thread Shea Parkes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003864#comment-16003864 ] Shea Parkes commented on SPARK-20683: - For anyone that found this issue and just wants to revert to

[jira] [Updated] (SPARK-20665) Spark-sql, "Bround" function return NULL

2017-05-09 Thread liuxian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuxian updated SPARK-20665: Description: >select bround(12.3, 2); >NULL For this case, the expected result is 12.3, but it is null

[jira] [Assigned] (SPARK-20686) PropagateEmptyRelation incorrectly handles aggregate without grouping expressions

2017-05-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20686: Assignee: Josh Rosen (was: Apache Spark) > PropagateEmptyRelation incorrectly handles

[jira] [Assigned] (SPARK-20686) PropagateEmptyRelation incorrectly handles aggregate without grouping expressions

2017-05-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20686: Assignee: Apache Spark (was: Josh Rosen) > PropagateEmptyRelation incorrectly handles

[jira] [Commented] (SPARK-20686) PropagateEmptyRelation incorrectly handles aggregate without grouping expressions

2017-05-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003848#comment-16003848 ] Apache Spark commented on SPARK-20686: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Created] (SPARK-20686) PropagateEmptyRelation incorrectly handles aggregate without grouping expressions

2017-05-09 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-20686: -- Summary: PropagateEmptyRelation incorrectly handles aggregate without grouping expressions Key: SPARK-20686 URL: https://issues.apache.org/jira/browse/SPARK-20686

[jira] [Commented] (SPARK-20311) SQL "range(N) as alias" or "range(N) alias" doesn't work

2017-05-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003841#comment-16003841 ] Apache Spark commented on SPARK-20311: -- User 'maropu' has created a pull request for this issue:

[jira] [Resolved] (SPARK-20500) ML, Graph 2.2 QA: API: Binary incompatible changes

2017-05-09 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-20500. --- Resolution: Done Fix Version/s: 2.2.0 > ML, Graph 2.2 QA: API: Binary

[jira] [Commented] (SPARK-20500) ML, Graph 2.2 QA: API: Binary incompatible changes

2017-05-09 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003813#comment-16003813 ] Joseph K. Bradley commented on SPARK-20500: --- I checked the full MiMa output for MLlib and

[jira] [Commented] (SPARK-20376) Make StateStoreProvider plugable

2017-05-09 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003729#comment-16003729 ] Michael Armbrust commented on SPARK-20376: -- /cc [~tdas] > Make StateStoreProvider plugable >

[jira] [Commented] (SPARK-20685) BatchPythonEvaluation UDF evaluator fails for case of single UDF with repeated argument

2017-05-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003662#comment-16003662 ] Apache Spark commented on SPARK-20685: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20685) BatchPythonEvaluation UDF evaluator fails for case of single UDF with repeated argument

2017-05-09 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-20685: -- Assignee: Josh Rosen > BatchPythonEvaluation UDF evaluator fails for case of single UDF with

[jira] [Created] (SPARK-20685) BatchPythonEvaluation UDF evaluator fails for case of single UDF with repeated argument

2017-05-09 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-20685: -- Summary: BatchPythonEvaluation UDF evaluator fails for case of single UDF with repeated argument Key: SPARK-20685 URL: https://issues.apache.org/jira/browse/SPARK-20685

[jira] [Assigned] (SPARK-20373) Batch queries with 'Dataset/DataFrame.withWatermark()` does not execute

2017-05-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-20373: Assignee: Genmao Yu > Batch queries with 'Dataset/DataFrame.withWatermark()` does not

[jira] [Updated] (SPARK-20373) Batch queries with 'Dataset/DataFrame.withWatermark()` does not execute

2017-05-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-20373: - Fix Version/s: 2.3.0 2.2.1 > Batch queries with

[jira] [Assigned] (SPARK-20311) SQL "range(N) as alias" or "range(N) alias" doesn't work

2017-05-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20311: Assignee: Apache Spark (was: Takeshi Yamamuro) > SQL "range(N) as alias" or "range(N)

[jira] [Assigned] (SPARK-20311) SQL "range(N) as alias" or "range(N) alias" doesn't work

2017-05-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20311: Assignee: Takeshi Yamamuro (was: Apache Spark) > SQL "range(N) as alias" or "range(N)

[jira] [Reopened] (SPARK-20311) SQL "range(N) as alias" or "range(N) alias" doesn't work

2017-05-09 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai reopened SPARK-20311: -- > SQL "range(N) as alias" or "range(N) alias" doesn't work >

[jira] [Commented] (SPARK-20311) SQL "range(N) as alias" or "range(N) alias" doesn't work

2017-05-09 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003626#comment-16003626 ] Yin Huai commented on SPARK-20311: -- It introduced a regression

[jira] [Updated] (SPARK-20311) SQL "range(N) as alias" or "range(N) alias" doesn't work

2017-05-09 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-20311: - Fix Version/s: (was: 2.2.1) (was: 2.3.0) > SQL "range(N) as alias" or

[jira] [Created] (SPARK-20684) expose createGlobalTempView in SparkR

2017-05-09 Thread Hossein Falaki (JIRA)
Hossein Falaki created SPARK-20684: -- Summary: expose createGlobalTempView in SparkR Key: SPARK-20684 URL: https://issues.apache.org/jira/browse/SPARK-20684 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-20205) DAGScheduler posts SparkListenerStageSubmitted before updating stage

2017-05-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20205: Assignee: Apache Spark > DAGScheduler posts SparkListenerStageSubmitted before updating

[jira] [Assigned] (SPARK-20205) DAGScheduler posts SparkListenerStageSubmitted before updating stage

2017-05-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20205: Assignee: (was: Apache Spark) > DAGScheduler posts SparkListenerStageSubmitted before

[jira] [Commented] (SPARK-20205) DAGScheduler posts SparkListenerStageSubmitted before updating stage

2017-05-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003471#comment-16003471 ] Apache Spark commented on SPARK-20205: -- User 'vanzin' has created a pull request for this issue:

[jira] [Commented] (SPARK-20683) Make table uncache chaining optional

2017-05-09 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003464#comment-16003464 ] Robert Kruszewski commented on SPARK-20683: --- +1, have the similar use case where we do cache

[jira] [Created] (SPARK-20683) Make table uncache chaining optional

2017-05-09 Thread Shea Parkes (JIRA)
Shea Parkes created SPARK-20683: --- Summary: Make table uncache chaining optional Key: SPARK-20683 URL: https://issues.apache.org/jira/browse/SPARK-20683 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-20682) Support a new faster ORC data source based on Apache ORC

2017-05-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20682: Assignee: Apache Spark > Support a new faster ORC data source based on Apache ORC >

[jira] [Assigned] (SPARK-20682) Support a new faster ORC data source based on Apache ORC

2017-05-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20682: Assignee: (was: Apache Spark) > Support a new faster ORC data source based on Apache

[jira] [Commented] (SPARK-20682) Support a new faster ORC data source based on Apache ORC

2017-05-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003369#comment-16003369 ] Apache Spark commented on SPARK-20682: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-20682) Support a new faster ORC data source based on Apache ORC

2017-05-09 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003344#comment-16003344 ] Dongjoon Hyun commented on SPARK-20682: --- Yes. I think it's worth for Spark to have it. > Support a

[jira] [Commented] (SPARK-20682) Support a new faster ORC data source based on Apache ORC

2017-05-09 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003339#comment-16003339 ] Robert Kruszewski commented on SPARK-20682: --- Awesome, didn't know that there are nohive

[jira] [Updated] (SPARK-20682) Support a new faster ORC data source based on Apache ORC

2017-05-09 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-20682: -- Description: Since SPARK-2883, Apache Spark supports Apache ORC inside `sql/hive` module with

[jira] [Comment Edited] (SPARK-20682) Support a new faster ORC data source based on Apache ORC

2017-05-09 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003326#comment-16003326 ] Dongjoon Hyun edited comment on SPARK-20682 at 5/9/17 7:10 PM: --- I fixed the

[jira] [Commented] (SPARK-20682) Support a new faster ORC data source based on Apache ORC

2017-05-09 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003326#comment-16003326 ] Dongjoon Hyun commented on SPARK-20682: --- Yep. That was a old huddle before 1.4.0. 1.4.0 provides

[jira] [Commented] (SPARK-20682) Support a new faster ORC data source based on Apache ORC

2017-05-09 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003321#comment-16003321 ] Robert Kruszewski commented on SPARK-20682: --- Would it make sense to bring hive-storage-api to

[jira] [Created] (SPARK-20682) Support a new faster ORC data source based on Apache ORC

2017-05-09 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-20682: - Summary: Support a new faster ORC data source based on Apache ORC Key: SPARK-20682 URL: https://issues.apache.org/jira/browse/SPARK-20682 Project: Spark

[jira] [Comment Edited] (SPARK-18528) limit + groupBy leads to java.lang.NullPointerException

2017-05-09 Thread Esther Kundin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003265#comment-16003265 ] Esther Kundin edited comment on SPARK-18528 at 5/9/17 6:43 PM: --- I have just

[jira] [Comment Edited] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

2017-05-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003267#comment-16003267 ] Reynold Xin edited comment on SPARK-12297 at 5/9/17 6:40 PM: - I looked at the

[jira] [Comment Edited] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

2017-05-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003267#comment-16003267 ] Reynold Xin edited comment on SPARK-12297 at 5/9/17 6:39 PM: - I looked at the

[jira] [Commented] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

2017-05-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003267#comment-16003267 ] Reynold Xin commented on SPARK-12297: - I looked at the issue again and reverted the patch. If we want

[jira] [Commented] (SPARK-18528) limit + groupBy leads to java.lang.NullPointerException

2017-05-09 Thread Esther Kundin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003265#comment-16003265 ] Esther Kundin commented on SPARK-18528: --- I have just found the same bug with Spark 2.1, even though

[jira] [Updated] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

2017-05-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-12297: Fix Version/s: (was: 2.3.0) > Add work-around for Parquet/Hive int96 timestamp bug. >

[jira] [Reopened] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

2017-05-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reopened SPARK-12297: - Assignee: (was: Imran Rashid) > Add work-around for Parquet/Hive int96 timestamp bug. >

[jira] [Comment Edited] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

2017-05-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001214#comment-16001214 ] Reynold Xin edited comment on SPARK-12297 at 5/9/17 6:35 PM: - Sorry I'm going

[jira] [Resolved] (SPARK-20627) Remove pip local version string (PEP440)

2017-05-09 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk resolved SPARK-20627. - Resolution: Fixed Fix Version/s: 2.1.2 2.3.0 2.2.1 Issue

[jira] [Assigned] (SPARK-20627) Remove pip local version string (PEP440)

2017-05-09 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk reassigned SPARK-20627: --- Assignee: holdenk > Remove pip local version string (PEP440) >

[jira] [Updated] (SPARK-20666) Flaky test - SparkListenerBus randomly failing java.lang.IllegalAccessError

2017-05-09 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-20666: --- Summary: Flaky test - SparkListenerBus randomly failing java.lang.IllegalAccessError (was:

[jira] [Commented] (SPARK-20666) Flaky test - SparkListenerBus randomly failing java.lang.IllegalAccessError on Windows

2017-05-09 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003116#comment-16003116 ] Marcelo Vanzin commented on SPARK-20666: I'm raising the severity because lots of PRs are failing

[jira] [Updated] (SPARK-20666) Flaky test - SparkListenerBus randomly failing java.lang.IllegalAccessError on Windows

2017-05-09 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-20666: --- Priority: Critical (was: Major) > Flaky test - SparkListenerBus randomly failing

[jira] [Commented] (SPARK-10408) Autoencoder

2017-05-09 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003110#comment-16003110 ] Alexander Ulanov commented on SPARK-10408: -- Autoencoder is implemented in the referenced pull

[jira] [Commented] (SPARK-14584) Improve recognition of non-nullability in Dataset transformations

2017-05-09 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003082#comment-16003082 ] Josh Rosen commented on SPARK-14584: [~hyukjin.kwon], yeah, this does appear to be fixed. My only

[jira] [Commented] (SPARK-20666) Flaky test - SparkListenerBus randomly failing java.lang.IllegalAccessError on Windows

2017-05-09 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003055#comment-16003055 ] Marcelo Vanzin commented on SPARK-20666: [~cloud_fan] do you think your fix for SPARK-12837 could

[jira] [Comment Edited] (SPARK-20666) Flaky test - SparkListenerBus randomly failing java.lang.IllegalAccessError on Windows

2017-05-09 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002962#comment-16002962 ] Marcelo Vanzin edited comment on SPARK-20666 at 5/9/17 4:52 PM: This

[jira] [Resolved] (SPARK-20674) Support registering UserDefinedFunction as named UDF

2017-05-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-20674. - Resolution: Fixed Fix Version/s: 2.2.0 > Support registering UserDefinedFunction as named UDF >

[jira] [Commented] (SPARK-20666) Flaky test - SparkListenerBus randomly failing java.lang.IllegalAccessError on Windows

2017-05-09 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002962#comment-16002962 ] Marcelo Vanzin commented on SPARK-20666: This doesn't fail just on Windows. It's been failing in

[jira] [Comment Edited] (SPARK-20666) Flaky test - SparkListenerBus randomly failing java.lang.IllegalAccessError on Windows

2017-05-09 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002962#comment-16002962 ] Marcelo Vanzin edited comment on SPARK-20666 at 5/9/17 4:18 PM: This

[jira] [Resolved] (SPARK-20548) Flaky Test: ReplSuite.newProductSeqEncoder with REPL defined class

2017-05-09 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20548. - Resolution: Fixed Fix Version/s: 2.3.0 2.2.1 Issue resolved by pull

[jira] [Commented] (SPARK-20608) Standby namenodes should be allowed to included in yarn.spark.access.namenodes to support HDFS HA

2017-05-09 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002956#comment-16002956 ] Marcelo Vanzin commented on SPARK-20608: bq. I think it is unreasonable that spark application

[jira] [Created] (SPARK-20681) DataFram.Drop doesn't take effective, neither does error

2017-05-09 Thread Xi Wang (JIRA)
Xi Wang created SPARK-20681: --- Summary: DataFram.Drop doesn't take effective, neither does error Key: SPARK-20681 URL: https://issues.apache.org/jira/browse/SPARK-20681 Project: Spark Issue Type:

[jira] [Commented] (SPARK-13747) Concurrent execution in SQL doesn't work with Scala ForkJoinPool

2017-05-09 Thread Dmitry Naumenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002852#comment-16002852 ] Dmitry Naumenko commented on SPARK-13747: - [~revolucion09] It's a bit an off-topic discussion. My

[jira] [Commented] (SPARK-13747) Concurrent execution in SQL doesn't work with Scala ForkJoinPool

2017-05-09 Thread Saif Addin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002814#comment-16002814 ] Saif Addin commented on SPARK-13747: [~dnaumenko] Nonetheless, if I am not mistaken, there are proofs

[jira] [Resolved] (SPARK-20355) Display Spark version on history page

2017-05-09 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-20355. --- Resolution: Fixed Assignee: Sanket Reddy Fix Version/s: 2.3.0 > Display

[jira] [Updated] (SPARK-20489) Different results in local mode and yarn mode when working with dates (silent corruption due to system timezone setting)

2017-05-09 Thread Rick Moritz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rick Moritz updated SPARK-20489: Summary: Different results in local mode and yarn mode when working with dates (silent corruption

[jira] [Commented] (SPARK-20489) Different results in local mode and yarn mode when working with dates (race condition with SimpleDateFormat?)

2017-05-09 Thread Rick Moritz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002762#comment-16002762 ] Rick Moritz commented on SPARK-20489: - Okay, I had a look at timezones, and it appears my Driver was

[jira] [Assigned] (SPARK-20591) Succeeded tasks num not equal in job page and job detail page on spark web ui when speculative task(s) exist

2017-05-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20591: Assignee: (was: Apache Spark) > Succeeded tasks num not equal in job page and job

[jira] [Assigned] (SPARK-20591) Succeeded tasks num not equal in job page and job detail page on spark web ui when speculative task(s) exist

2017-05-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20591: Assignee: Apache Spark > Succeeded tasks num not equal in job page and job detail page on

[jira] [Commented] (SPARK-20591) Succeeded tasks num not equal in job page and job detail page on spark web ui when speculative task(s) exist

2017-05-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002686#comment-16002686 ] Apache Spark commented on SPARK-20591: -- User 'fjh100456' has created a pull request for this issue:

[jira] [Commented] (SPARK-13747) Concurrent execution in SQL doesn't work with Scala ForkJoinPool

2017-05-09 Thread Dmitry Naumenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002674#comment-16002674 ] Dmitry Naumenko commented on SPARK-13747: - [~zsxwing] I've tried to build a Spark from a branch

[jira] [Commented] (SPARK-20542) Add an API into Bucketizer that can bin a lot of columns all at once

2017-05-09 Thread Barry Becker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002609#comment-16002609 ] Barry Becker commented on SPARK-20542: -- This is a great improvement, @viirya! According to your

[jira] [Assigned] (SPARK-20311) SQL "range(N) as alias" or "range(N) alias" doesn't work

2017-05-09 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20311: --- Assignee: Takeshi Yamamuro > SQL "range(N) as alias" or "range(N) alias" doesn't work >

[jira] [Resolved] (SPARK-20311) SQL "range(N) as alias" or "range(N) alias" doesn't work

2017-05-09 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20311. - Resolution: Fixed Fix Version/s: 2.3.0 2.2.1 Issue resolved by pull

[jira] [Commented] (SPARK-19581) running NaiveBayes model with 0 features can crash the executor with D rorreGEMV

2017-05-09 Thread Barry Becker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002582#comment-16002582 ] Barry Becker commented on SPARK-19581: -- I think its just a matter of sending a feature vector of

[jira] [Resolved] (SPARK-20667) Cleanup the cataloged metadata after completing the package of sql/core and sql/hive

2017-05-09 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20667. - Resolution: Fixed Fix Version/s: 2.3.0 2.2.1 Issue resolved by pull

[jira] [Assigned] (SPARK-20601) Python API Changes for Constrained Logistic Regression Params

2017-05-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20601: Assignee: Apache Spark > Python API Changes for Constrained Logistic Regression Params >

[jira] [Assigned] (SPARK-20601) Python API Changes for Constrained Logistic Regression Params

2017-05-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20601: Assignee: (was: Apache Spark) > Python API Changes for Constrained Logistic

[jira] [Commented] (SPARK-20601) Python API Changes for Constrained Logistic Regression Params

2017-05-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002493#comment-16002493 ] Apache Spark commented on SPARK-20601: -- User 'zero323' has created a pull request for this issue:

[jira] [Commented] (SPARK-2060) Querying JSON Datasets with SQL and DSL in Spark SQL

2017-05-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002490#comment-16002490 ] Apache Spark commented on SPARK-2060: - User 'zero323' has created a pull request for this issue:

[jira] [Updated] (SPARK-20680) Spark-sql do not support for void column datatype of view

2017-05-09 Thread Lantao Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin updated SPARK-20680: --- Summary: Spark-sql do not support for void column datatype of view (was: Spark-sql do not support

[jira] [Created] (SPARK-20680) Spark-sql do not support for column datatype of void in a HIVE view

2017-05-09 Thread Lantao Jin (JIRA)
Lantao Jin created SPARK-20680: -- Summary: Spark-sql do not support for column datatype of void in a HIVE view Key: SPARK-20680 URL: https://issues.apache.org/jira/browse/SPARK-20680 Project: Spark

[jira] [Updated] (SPARK-19364) Stream Blocks in Storage Persists Forever when Kinesis Checkpoints are enabled and an exception is thrown

2017-05-09 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-19364: - Priority: Major (was: Blocker) I am lowering the priority to {{Major}} as higher priorities

[jira] [Updated] (SPARK-20415) SPARK job hangs while writing DataFrame to HDFS

2017-05-09 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-20415: - Priority: Major (was: Blocker) I am lowering the priority to {{Major}} as higher priorities

[jira] [Commented] (SPARK-19876) Add OneTime trigger executor

2017-05-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002393#comment-16002393 ] Apache Spark commented on SPARK-19876: -- User 'srowen' has created a pull request for this issue:

[jira] [Commented] (SPARK-20554) Remove usage of scala.language.reflectiveCalls

2017-05-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16002378#comment-16002378 ] Sean Owen commented on SPARK-20554: --- [~umesh9...@gmail.com] are you working on this? > Remove usage of

[jira] [Resolved] (SPARK-20580) Allow RDD cache with unserializable objects

2017-05-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20580. --- Resolution: Not A Problem Although map() wouldn't generally entail serializing anything, it could

[jira] [Resolved] (SPARK-20606) ML 2.2 QA: Remove deprecated methods for ML

2017-05-09 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang resolved SPARK-20606. - Resolution: Fixed Fix Version/s: 2.2.0 > ML 2.2 QA: Remove deprecated methods for ML >

  1   2   >