[jira] [Assigned] (SPARK-24132) Instruments improvement for classification

[ https://issues.apache.org/jira/browse/SPARK-24132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24132: Assignee: (was: Apache Spark) > Instruments improvement for classification > -

[jira] [Commented] (SPARK-24132) Instruments improvement for classification

[ https://issues.apache.org/jira/browse/SPARK-24132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459477#comment-16459477 ] Apache Spark commented on SPARK-24132: -- User 'ludatabricks' has created a pull reque

[jira] [Assigned] (SPARK-24132) Instruments improvement for classification

[ https://issues.apache.org/jira/browse/SPARK-24132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24132: Assignee: Apache Spark > Instruments improvement for classification >

[jira] [Created] (SPARK-24132) Instruments improvement for classification

Lu Wang created SPARK-24132: --- Summary: Instruments improvement for classification Key: SPARK-24132 URL: https://issues.apache.org/jira/browse/SPARK-24132 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-23342) Add ORC configuration tests for ORC data source

[ https://issues.apache.org/jira/browse/SPARK-23342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-23342. --- Resolution: Not A Problem I close this issue as a `Not A Problem`. > Add ORC configuration t

[jira] [Commented] (SPARK-24131) Add majorMinorVersion API to PySpark for determining Spark versions

[ https://issues.apache.org/jira/browse/SPARK-24131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459427#comment-16459427 ] Apache Spark commented on SPARK-24131: -- User 'viirya' has created a pull request for

[jira] [Assigned] (SPARK-24131) Add majorMinorVersion API to PySpark for determining Spark versions

[ https://issues.apache.org/jira/browse/SPARK-24131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24131: Assignee: Apache Spark > Add majorMinorVersion API to PySpark for determining Spark versio

[jira] [Assigned] (SPARK-24131) Add majorMinorVersion API to PySpark for determining Spark versions

[ https://issues.apache.org/jira/browse/SPARK-24131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24131: Assignee: (was: Apache Spark) > Add majorMinorVersion API to PySpark for determining S

[jira] [Created] (SPARK-24131) Add majorMinorVersion API to PySpark for determining Spark versions

Liang-Chi Hsieh created SPARK-24131: --- Summary: Add majorMinorVersion API to PySpark for determining Spark versions Key: SPARK-24131 URL: https://issues.apache.org/jira/browse/SPARK-24131 Project: Sp

[jira] [Created] (SPARK-24130) Data Source V2: Join Push Down

Jia Li created SPARK-24130: -- Summary: Data Source V2: Join Push Down Key: SPARK-24130 URL: https://issues.apache.org/jira/browse/SPARK-24130 Project: Spark Issue Type: Sub-task Components:

[jira] [Resolved] (SPARK-23853) Skip doctests which require hive support built in PySpark

[ https://issues.apache.org/jira/browse/SPARK-23853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23853. -- Resolution: Fixed Fix Version/s: 2.3.1 2.4.0 Issue resolved by pull r

[jira] [Assigned] (SPARK-23853) Skip doctests which require hive support built in PySpark

[ https://issues.apache.org/jira/browse/SPARK-23853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-23853: Assignee: Dongjoon Hyun > Skip doctests which require hive support built in PySpark >

[jira] [Assigned] (SPARK-24129) Add option to pass --build-arg's to docker-image-tool.sh

[ https://issues.apache.org/jira/browse/SPARK-24129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24129: Assignee: (was: Apache Spark) > Add option to pass --build-arg's to docker-image-tool.

[jira] [Commented] (SPARK-24129) Add option to pass --build-arg's to docker-image-tool.sh

[ https://issues.apache.org/jira/browse/SPARK-24129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459349#comment-16459349 ] Apache Spark commented on SPARK-24129: -- User 'devaraj-kavali' has created a pull req

[jira] [Assigned] (SPARK-24129) Add option to pass --build-arg's to docker-image-tool.sh

[ https://issues.apache.org/jira/browse/SPARK-24129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24129: Assignee: Apache Spark > Add option to pass --build-arg's to docker-image-tool.sh > --

[jira] [Updated] (SPARK-23940) High-order function: transform_values(map, function) → map

[ https://issues.apache.org/jira/browse/SPARK-23940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henry Robinson updated SPARK-23940: --- Summary: High-order function: transform_values(map, function) → map (was: High-ofer function:

[jira] [Commented] (SPARK-24128) Mention spark.sql.crossJoin.enabled in implicit cartesian product error msg

[ https://issues.apache.org/jira/browse/SPARK-24128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459333#comment-16459333 ] Apache Spark commented on SPARK-24128: -- User 'henryr' has created a pull request for

[jira] [Assigned] (SPARK-24128) Mention spark.sql.crossJoin.enabled in implicit cartesian product error msg

[ https://issues.apache.org/jira/browse/SPARK-24128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24128: Assignee: (was: Apache Spark) > Mention spark.sql.crossJoin.enabled in implicit cartes

[jira] [Assigned] (SPARK-24128) Mention spark.sql.crossJoin.enabled in implicit cartesian product error msg

[ https://issues.apache.org/jira/browse/SPARK-24128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24128: Assignee: Apache Spark > Mention spark.sql.crossJoin.enabled in implicit cartesian product

[jira] [Created] (SPARK-24129) Add option to pass --build-arg's to docker-image-tool.sh

Devaraj K created SPARK-24129: - Summary: Add option to pass --build-arg's to docker-image-tool.sh Key: SPARK-24129 URL: https://issues.apache.org/jira/browse/SPARK-24129 Project: Spark Issue Type

[jira] [Created] (SPARK-24128) Mention spark.sql.crossJoin.enabled in implicit cartesian product error msg

Henry Robinson created SPARK-24128: -- Summary: Mention spark.sql.crossJoin.enabled in implicit cartesian product error msg Key: SPARK-24128 URL: https://issues.apache.org/jira/browse/SPARK-24128 Proje

[jira] [Assigned] (SPARK-24039) remove restarting iterators hack

[ https://issues.apache.org/jira/browse/SPARK-24039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24039: Assignee: Apache Spark > remove restarting iterators hack > --

[jira] [Assigned] (SPARK-24039) remove restarting iterators hack

[ https://issues.apache.org/jira/browse/SPARK-24039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24039: Assignee: (was: Apache Spark) > remove restarting iterators hack > ---

[jira] [Commented] (SPARK-24039) remove restarting iterators hack

[ https://issues.apache.org/jira/browse/SPARK-24039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459208#comment-16459208 ] Apache Spark commented on SPARK-24039: -- User 'jose-torres' has created a pull reques

[jira] [Commented] (SPARK-23549) Spark SQL unexpected behavior when comparing timestamp to date

[ https://issues.apache.org/jira/browse/SPARK-23549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459136#comment-16459136 ] Dongjoon Hyun commented on SPARK-23549: --- [~emlyn]. This issue changes the behavior

[jira] [Assigned] (SPARK-24127) Support text socket source in continuous mode

[ https://issues.apache.org/jira/browse/SPARK-24127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24127: Assignee: (was: Apache Spark) > Support text socket source in continuous mode > --

[jira] [Assigned] (SPARK-24127) Support text socket source in continuous mode

[ https://issues.apache.org/jira/browse/SPARK-24127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24127: Assignee: Apache Spark > Support text socket source in continuous mode > -

[jira] [Commented] (SPARK-24127) Support text socket source in continuous mode

[ https://issues.apache.org/jira/browse/SPARK-24127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459129#comment-16459129 ] Apache Spark commented on SPARK-24127: -- User 'arunmahadevan' has created a pull requ

[jira] [Created] (SPARK-24127) Support text socket source in continuous mode

Arun Mahadevan created SPARK-24127: -- Summary: Support text socket source in continuous mode Key: SPARK-24127 URL: https://issues.apache.org/jira/browse/SPARK-24127 Project: Spark Issue Type:

[jira] [Commented] (SPARK-24115) improve instrumentation for spark.ml.tuning

[ https://issues.apache.org/jira/browse/SPARK-24115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459033#comment-16459033 ] Joseph K. Bradley commented on SPARK-24115: --- Sounds good; go ahead. > improve

[jira] [Assigned] (SPARK-24003) Add support to provide spark.executor.extraJavaOptions in terms of App Id and/or Executor Id's

[ https://issues.apache.org/jira/browse/SPARK-24003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-24003: -- Assignee: Devaraj K > Add support to provide spark.executor.extraJavaOptions in terms

[jira] [Resolved] (SPARK-24003) Add support to provide spark.executor.extraJavaOptions in terms of App Id and/or Executor Id's

[ https://issues.apache.org/jira/browse/SPARK-24003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-24003. Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21088 [https:/

[jira] [Updated] (SPARK-23781) Merge YARN and Mesos token renewal code

[ https://issues.apache.org/jira/browse/SPARK-23781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-23781: --- Component/s: (was: yan) YARN > Merge YARN and Mesos token renewal code >

[jira] [Commented] (SPARK-24126) PySpark tests leave a lot of garbage in /tmp

[ https://issues.apache.org/jira/browse/SPARK-24126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459000#comment-16459000 ] Apache Spark commented on SPARK-24126: -- User 'vanzin' has created a pull request for

[jira] [Assigned] (SPARK-24126) PySpark tests leave a lot of garbage in /tmp

[ https://issues.apache.org/jira/browse/SPARK-24126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24126: Assignee: Apache Spark > PySpark tests leave a lot of garbage in /tmp > --

[jira] [Assigned] (SPARK-24126) PySpark tests leave a lot of garbage in /tmp

[ https://issues.apache.org/jira/browse/SPARK-24126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24126: Assignee: (was: Apache Spark) > PySpark tests leave a lot of garbage in /tmp > ---

[jira] [Commented] (SPARK-24126) PySpark tests leave a lot of garbage in /tmp

[ https://issues.apache.org/jira/browse/SPARK-24126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16458974#comment-16458974 ] Marcelo Vanzin commented on SPARK-24126: BTW I'm testing changes to implement thi

[jira] [Created] (SPARK-24126) PySpark tests leave a lot of garbage in /tmp

Marcelo Vanzin created SPARK-24126: -- Summary: PySpark tests leave a lot of garbage in /tmp Key: SPARK-24126 URL: https://issues.apache.org/jira/browse/SPARK-24126 Project: Spark Issue Type:

[jira] [Created] (SPARK-24125) Add quoting rules to SQL guide

Henry Robinson created SPARK-24125: -- Summary: Add quoting rules to SQL guide Key: SPARK-24125 URL: https://issues.apache.org/jira/browse/SPARK-24125 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-23971) Should not leak Spark sessions across test suites

[ https://issues.apache.org/jira/browse/SPARK-23971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16458959#comment-16458959 ] Apache Spark commented on SPARK-23971: -- User 'gatorsmile' has created a pull request

[jira] [Assigned] (SPARK-24123) Fix a flaky test `DateTimeUtilsSuite.monthsBetween`

[ https://issues.apache.org/jira/browse/SPARK-24123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24123: Assignee: (was: Apache Spark) > Fix a flaky test `DateTimeUtilsSuite.monthsBetween` >

[jira] [Commented] (SPARK-24123) Fix a flaky test `DateTimeUtilsSuite.monthsBetween`

[ https://issues.apache.org/jira/browse/SPARK-24123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16458907#comment-16458907 ] Apache Spark commented on SPARK-24123: -- User 'mgaido91' has created a pull request f

[jira] [Assigned] (SPARK-24123) Fix a flaky test `DateTimeUtilsSuite.monthsBetween`

[ https://issues.apache.org/jira/browse/SPARK-24123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24123: Assignee: Apache Spark > Fix a flaky test `DateTimeUtilsSuite.monthsBetween` > ---

[jira] [Commented] (SPARK-24124) Spark history server should create spark.history.store.path and set permissions properly

[ https://issues.apache.org/jira/browse/SPARK-24124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16458904#comment-16458904 ] Marcelo Vanzin commented on SPARK-24124: This should be fine. > Spark history se

[jira] [Commented] (SPARK-23530) It's not appropriate to let the original master exit while the leader of zookeeper shutdown

[ https://issues.apache.org/jira/browse/SPARK-23530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16458895#comment-16458895 ] Ashwin Agate commented on SPARK-23530: --  Can we please increase the priority of this

[jira] [Comment Edited] (SPARK-15544) Bouncing Zookeeper node causes Active spark master to exit

[ https://issues.apache.org/jira/browse/SPARK-15544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1645#comment-1645 ] Ashwin Agate edited comment on SPARK-15544 at 4/30/18 7:05 PM:

[jira] [Comment Edited] (SPARK-15544) Bouncing Zookeeper node causes Active spark master to exit

[ https://issues.apache.org/jira/browse/SPARK-15544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1645#comment-1645 ] Ashwin Agate edited comment on SPARK-15544 at 4/30/18 7:05 PM:

[jira] [Comment Edited] (SPARK-15544) Bouncing Zookeeper node causes Active spark master to exit

[ https://issues.apache.org/jira/browse/SPARK-15544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1645#comment-1645 ] Ashwin Agate edited comment on SPARK-15544 at 4/30/18 7:05 PM:

[jira] [Commented] (SPARK-15544) Bouncing Zookeeper node causes Active spark master to exit

[ https://issues.apache.org/jira/browse/SPARK-15544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1645#comment-1645 ] agate commented on SPARK-15544: --- Can we please increase the priority of this bug since it e

[jira] [Commented] (SPARK-24124) Spark history server should create spark.history.store.path and set permissions properly

[ https://issues.apache.org/jira/browse/SPARK-24124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16458835#comment-16458835 ] Thomas Graves commented on SPARK-24124: --- [~vanzin]  any objections to this? > Spar

[jira] [Created] (SPARK-24124) Spark history server should create spark.history.store.path and set permissions properly

Thomas Graves created SPARK-24124: - Summary: Spark history server should create spark.history.store.path and set permissions properly Key: SPARK-24124 URL: https://issues.apache.org/jira/browse/SPARK-24124

[jira] [Created] (SPARK-24123) Fix a flaky test `DateTimeUtilsSuite.monthsBetween`

Dongjoon Hyun created SPARK-24123: - Summary: Fix a flaky test `DateTimeUtilsSuite.monthsBetween` Key: SPARK-24123 URL: https://issues.apache.org/jira/browse/SPARK-24123 Project: Spark Issue T

[jira] [Commented] (SPARK-23993) Support DESC FORMATTED table_name column_name

[ https://issues.apache.org/jira/browse/SPARK-23993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16458822#comment-16458822 ] Sunitha Kambhampati commented on SPARK-23993: - I just tried it out in the cur

[jira] [Commented] (SPARK-23975) Allow Clustering to take Arrays of Double as input features

[ https://issues.apache.org/jira/browse/SPARK-23975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16458778#comment-16458778 ] Apache Spark commented on SPARK-23975: -- User 'ludatabricks' has created a pull reque

[jira] [Resolved] (SPARK-24072) clearly define pushed filters

[ https://issues.apache.org/jira/browse/SPARK-24072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24072. - Resolution: Fixed Fix Version/s: 2.4.0 > clearly define pushed filters > -

[jira] [Created] (SPARK-24122) Allow automatic driver restarts on K8s

Oz Ben-Ami created SPARK-24122: -- Summary: Allow automatic driver restarts on K8s Key: SPARK-24122 URL: https://issues.apache.org/jira/browse/SPARK-24122 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-24046) Rate Source doesn't gradually increase rate when rampUpTime>=RowsPerSecond

[ https://issues.apache.org/jira/browse/SPARK-24046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16458636#comment-16458636 ] Apache Spark commented on SPARK-24046: -- User 'maasg' has created a pull request for

[jira] [Assigned] (SPARK-24121) The API for handling expression code generation in expression codegen

[ https://issues.apache.org/jira/browse/SPARK-24121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24121: Assignee: Apache Spark > The API for handling expression code generation in expression cod

[jira] [Assigned] (SPARK-24121) The API for handling expression code generation in expression codegen

[ https://issues.apache.org/jira/browse/SPARK-24121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24121: Assignee: (was: Apache Spark) > The API for handling expression code generation in exp

[jira] [Commented] (SPARK-24121) The API for handling expression code generation in expression codegen

[ https://issues.apache.org/jira/browse/SPARK-24121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16458574#comment-16458574 ] Apache Spark commented on SPARK-24121: -- User 'viirya' has created a pull request for

[jira] [Created] (SPARK-24121) The API for handling expression code generation in expression codegen

Liang-Chi Hsieh created SPARK-24121: --- Summary: The API for handling expression code generation in expression codegen Key: SPARK-24121 URL: https://issues.apache.org/jira/browse/SPARK-24121 Project:

[jira] [Commented] (SPARK-23549) Spark SQL unexpected behavior when comparing timestamp to date

[ https://issues.apache.org/jira/browse/SPARK-23549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16458511#comment-16458511 ] Emlyn Corrin commented on SPARK-23549: -- Will this be included in Spark 2.3.1? It onl