[jira] [Created] (SPARK-23460) PySpark concurrency python egg cache directory

2018-02-17 Thread Dmitiry (JIRA)
Dmitiry created SPARK-23460: --- Summary: PySpark concurrency python egg cache directory Key: SPARK-23460 URL: https://issues.apache.org/jira/browse/SPARK-23460 Project: Spark Issue Type: Question

[jira] [Commented] (SPARK-18778) Fix the Scala classpath in the spark-shell

2018-02-17 Thread Shaun Jackman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368442#comment-16368442 ] Shaun Jackman commented on SPARK-18778: --- I have this issue as well. spark-shell fails with Java 9.

[jira] [Comment Edited] (SPARK-18778) Fix the Scala classpath in the spark-shell

2018-02-17 Thread Shaun Jackman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368442#comment-16368442 ] Shaun Jackman edited comment on SPARK-18778 at 2/18/18 5:25 AM: I have

[jira] [Updated] (SPARK-23459) Improve the error message when unknown column is specified in partition columns

2018-02-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-23459: Description: {noformat} test("save with an unknown partition column") { withTempDir { dir =>

[jira] [Created] (SPARK-23459) Improve the error message when unknown column is specified in partition columns

2018-02-17 Thread Xiao Li (JIRA)
Xiao Li created SPARK-23459: --- Summary: Improve the error message when unknown column is specified in partition columns Key: SPARK-23459 URL: https://issues.apache.org/jira/browse/SPARK-23459 Project: Spark

[jira] [Commented] (SPARK-23435) R tests should support latest testthat

2018-02-17 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368351#comment-16368351 ] Felix Cheung commented on SPARK-23435: -- Working on this. Debugging a problem. > R tests should

[jira] [Commented] (SPARK-23423) Application declines any offers when killed+active executors rich spark.dynamicAllocation.maxExecutors

2018-02-17 Thread Susan X. Huynh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368331#comment-16368331 ] Susan X. Huynh commented on SPARK-23423: [~skonto] [~igor.berman] I got some info from a

[jira] [Commented] (SPARK-23458) OrcSuite flaky test

2018-02-17 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368295#comment-16368295 ] Marco Gaido commented on SPARK-23458: - cc [~dongjoon] > OrcSuite flaky test > --- >

[jira] [Created] (SPARK-23458) OrcSuite flaky test

2018-02-17 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-23458: --- Summary: OrcSuite flaky test Key: SPARK-23458 URL: https://issues.apache.org/jira/browse/SPARK-23458 Project: Spark Issue Type: Task Components: SQL

[jira] [Commented] (SPARK-23457) Register task completion listeners first for ParquetFileFormat

2018-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368291#comment-16368291 ] Apache Spark commented on SPARK-23457: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-23457) Register task completion listeners first for ParquetFileFormat

2018-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23457: Assignee: (was: Apache Spark) > Register task completion listeners first for

[jira] [Assigned] (SPARK-23457) Register task completion listeners first for ParquetFileFormat

2018-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23457: Assignee: Apache Spark > Register task completion listeners first for ParquetFileFormat >

[jira] [Updated] (SPARK-23457) Register task completion listeners first for Parquet

2018-02-17 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23457: -- Description: ParquetFileFormat leaks open files in some cases. This issue aims to register

[jira] [Updated] (SPARK-23457) Register task completion listeners first for ParquetFileFormat

2018-02-17 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23457: -- Summary: Register task completion listeners first for ParquetFileFormat (was: Register task

[jira] [Created] (SPARK-23457) Register task completion listeners first for Parquet

2018-02-17 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-23457: - Summary: Register task completion listeners first for Parquet Key: SPARK-23457 URL: https://issues.apache.org/jira/browse/SPARK-23457 Project: Spark Issue

[jira] [Assigned] (SPARK-21783) Turn on ORC filter push-down by default

2018-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21783: Assignee: Dongjoon Hyun (was: Apache Spark) > Turn on ORC filter push-down by default >

[jira] [Assigned] (SPARK-23456) Turn on `native` ORC implementation by default

2018-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23456: Assignee: (was: Apache Spark) > Turn on `native` ORC implementation by default >

[jira] [Commented] (SPARK-21783) Turn on ORC filter push-down by default

2018-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368289#comment-16368289 ] Apache Spark commented on SPARK-21783: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-21783) Turn on ORC filter push-down by default

2018-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21783: Assignee: Apache Spark (was: Dongjoon Hyun) > Turn on ORC filter push-down by default >

[jira] [Assigned] (SPARK-23456) Turn on `native` ORC implementation by default

2018-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23456: Assignee: Apache Spark > Turn on `native` ORC implementation by default >

[jira] [Commented] (SPARK-23456) Turn on `native` ORC implementation by default

2018-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368288#comment-16368288 ] Apache Spark commented on SPARK-23456: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Created] (SPARK-23456) Turn on `native` ORC implementation by default

2018-02-17 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-23456: - Summary: Turn on `native` ORC implementation by default Key: SPARK-23456 URL: https://issues.apache.org/jira/browse/SPARK-23456 Project: Spark Issue Type:

[jira] [Commented] (SPARK-23423) Application declines any offers when killed+active executors rich spark.dynamicAllocation.maxExecutors

2018-02-17 Thread Susan X. Huynh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368261#comment-16368261 ] Susan X. Huynh commented on SPARK-23423: I'll check if the task updates might be dropped under

[jira] [Commented] (SPARK-23437) [ML] Distributed Gaussian Process Regression for MLlib

2018-02-17 Thread Valeriy Avanesov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368198#comment-16368198 ] Valeriy Avanesov commented on SPARK-23437: -- [~sethah], thanks for your input. I believe, GPflow

[jira] [Assigned] (SPARK-23455) Default Params in ML should be saved separately

2018-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23455: Assignee: Apache Spark > Default Params in ML should be saved separately >

[jira] [Assigned] (SPARK-23455) Default Params in ML should be saved separately

2018-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23455: Assignee: (was: Apache Spark) > Default Params in ML should be saved separately >

[jira] [Commented] (SPARK-23455) Default Params in ML should be saved separately

2018-02-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368154#comment-16368154 ] Apache Spark commented on SPARK-23455: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-23368) Avoid unnecessary Exchange or Sort after projection

2018-02-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368147#comment-16368147 ] Xiao Li commented on SPARK-23368: - [~maryannxue] Thank you for your work! Happy New Year! > Avoid

[jira] [Updated] (SPARK-23340) Upgrade Apache ORC to 1.4.3

2018-02-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-23340: Fix Version/s: (was: 2.3.0) 2.4.0 > Upgrade Apache ORC to 1.4.3 >