[jira] [Resolved] (SPARK-33215) Speed up event log download by skipping UI rebuild

2020-10-26 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-33215. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30126

[jira] [Assigned] (SPARK-33215) Speed up event log download by skipping UI rebuild

2020-10-26 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-33215: Assignee: Baohe Zhang > Speed up event log download by skipping UI rebuild >

[jira] [Resolved] (SPARK-33243) Add numpydoc into documentation dependency

2020-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33243. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30149

[jira] [Assigned] (SPARK-33243) Add numpydoc into documentation dependency

2020-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33243: Assignee: Hyukjin Kwon > Add numpydoc into documentation dependency >

[jira] [Created] (SPARK-33256) Update contribution guide about NumPy documentation style

2020-10-26 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-33256: Summary: Update contribution guide about NumPy documentation style Key: SPARK-33256 URL: https://issues.apache.org/jira/browse/SPARK-33256 Project: Spark

[jira] [Updated] (SPARK-33249) Add status plugin for live application

2020-10-26 Thread Weiyi Kong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiyi Kong updated SPARK-33249: --- Remaining Estimate: (was: 24h) Original Estimate: (was: 24h) > Add status plugin for

[jira] [Commented] (SPARK-33255) Use new API to construct ParquetFileReader and read Parquet footer

2020-10-26 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221126#comment-17221126 ] Yang Jie commented on SPARK-33255: -- [~hyukjin.kwon] Got it ~ > Use new API to construct

[jira] [Updated] (SPARK-33249) Add status plugin for live application

2020-10-26 Thread Weiyi Kong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiyi Kong updated SPARK-33249: --- Remaining Estimate: 24h Original Estimate: 24h > Add status plugin for live application >

[jira] [Commented] (SPARK-33250) Migration to NumPy documentation style in SQL (pyspark.sql.*)

2020-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221122#comment-17221122 ] Hyukjin Kwon commented on SPARK-33250: -- I'll work on this one. > Migration to NumPy documentation

[jira] [Updated] (SPARK-33255) Use new API to construct ParquetFileReader and read Parquet footer

2020-10-26 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-33255: - Description: {code:java} /** * @param configuration the Hadoop conf * @param fileMetaData

[jira] [Updated] (SPARK-33255) Use new API to construct ParquetFileReader and read Parquet footer

2020-10-26 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-33255: - Description: {code:java} /** * @param configuration the Hadoop conf * @param fileMetaData

[jira] [Updated] (SPARK-33255) Use new API to construct ParquetFileReader and read Parquet footer

2020-10-26 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-33255: - Description: {code:java} /** * @param configuration the Hadoop conf * @param fileMetaData

[jira] [Resolved] (SPARK-33255) Use new API to construct ParquetFileReader and read Parquet footer

2020-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33255. -- Resolution: Duplicate > Use new API to construct ParquetFileReader and read Parquet footer >

[jira] [Commented] (SPARK-33255) Use new API to construct ParquetFileReader and read Parquet footer

2020-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221119#comment-17221119 ] Hyukjin Kwon commented on SPARK-33255: -- We can't replace this now. See also

[jira] [Updated] (SPARK-33255) Use new API to construct ParquetFileReader and read Parquet footer

2020-10-26 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-33255: - Description: {code:java} /** * @param configuration the Hadoop conf * @param fileMetaData

[jira] [Updated] (SPARK-32085) Migrate to NumPy documentation style

2020-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32085: - Description: https://github.com/numpy/numpydoc For example, Before:

[jira] [Updated] (SPARK-33254) Migration to NumPy documentation style in Core (pyspark.*, pyspark.resource.*, etc.)

2020-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33254: - Description: This JIRA targets to migrate to NumPy documentation style in > Migration to NumPy

[jira] [Updated] (SPARK-33251) Migration to NumPy documentation style in ML (pyspark.ml.*)

2020-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33251: - Description: This JIRA targets to migrate to NumPy documentation style in MLlib

[jira] [Updated] (SPARK-33250) Migration to NumPy documentation style in SQL (pyspark.sql.*)

2020-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33250: - Description: Migration to NumPy documentation style in ML (pyspark.ml.*) > Migration to NumPy

[jira] [Updated] (SPARK-33251) Migration to NumPy documentation style in ML (pyspark.ml.*)

2020-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33251: - Description: This JIRA targets to migrate to NumPy documentation style in ML (pyspark.ml.*).

[jira] [Commented] (SPARK-32085) Migrate to NumPy documentation style

2020-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221099#comment-17221099 ] Hyukjin Kwon commented on SPARK-32085: -- cc [~zero323] in case you're interested in some of

[jira] [Updated] (SPARK-33250) Migration to NumPy documentation style in SQL (pyspark.sql.*)

2020-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33250: - Description: Migration to NumPy documentation style in SQL (pyspark.sql.*) (was: Migration to

[jira] [Updated] (SPARK-33254) Migration to NumPy documentation style in Core (pyspark.*, pyspark.resource.*, etc.)

2020-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33254: - Description: This JIRA targets to migrate to NumPy documentation style in Core (pyspark.\*,

[jira] [Commented] (SPARK-33246) Spark SQL null semantics documentation is incorrect

2020-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221116#comment-17221116 ] Hyukjin Kwon commented on SPARK-33246: -- [~stwhit] Apache Spark uses pull requests in GitHub to

[jira] [Updated] (SPARK-33253) Migration to NumPy documentation style in Streaming (pyspark.streaming.*)

2020-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33253: - Description: This JIRA targets to migrate to NumPy documentation style in Streaming

[jira] [Updated] (SPARK-33253) Migration to NumPy documentation style in Streaming (pyspark.streaming.*)

2020-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33253: - Description: This JIRA targets to migrate to NumPy documentation style in Core (pyspark.*,

[jira] [Updated] (SPARK-33252) Migration to NumPy documentation style in MLlib (pyspark.mllib.*)

2020-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33252: - Description: This JIRA targets to migrate to NumPy documentation style in MLlib

[jira] [Updated] (SPARK-33254) Migration to NumPy documentation style in Core (pyspark.*, pyspark.resource.*, etc.)

2020-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33254: - Description: This JIRA targets to migrate to NumPy documentation style in Core (pyspark.*,

[jira] [Updated] (SPARK-33252) Migration to NumPy documentation style in MLlib (pyspark.mllib.*)

2020-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33252: - Description: This JIRA targets to migrate to NumPy documentation style in Streaming

[jira] [Created] (SPARK-33254) Migration to NumPy documentation style in Core (pyspark.*, pyspark.resource.*, etc.)

2020-10-26 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-33254: Summary: Migration to NumPy documentation style in Core (pyspark.*, pyspark.resource.*, etc.) Key: SPARK-33254 URL: https://issues.apache.org/jira/browse/SPARK-33254

[jira] [Created] (SPARK-33252) Migration to NumPy documentation style in MLlib (pyspark.mllib.*)

2020-10-26 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-33252: Summary: Migration to NumPy documentation style in MLlib (pyspark.mllib.*) Key: SPARK-33252 URL: https://issues.apache.org/jira/browse/SPARK-33252 Project: Spark

[jira] [Created] (SPARK-33253) Migration to NumPy documentation style in Streaming (pyspark.streaming.*)

2020-10-26 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-33253: Summary: Migration to NumPy documentation style in Streaming (pyspark.streaming.*) Key: SPARK-33253 URL: https://issues.apache.org/jira/browse/SPARK-33253 Project:

[jira] [Created] (SPARK-33250) Migration to NumPy documentation style in SQL (pyspark.sql.*)

2020-10-26 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-33250: Summary: Migration to NumPy documentation style in SQL (pyspark.sql.*) Key: SPARK-33250 URL: https://issues.apache.org/jira/browse/SPARK-33250 Project: Spark

[jira] [Created] (SPARK-33251) Migration to NumPy documentation style in ML (pyspark.ml.*)

2020-10-26 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-33251: Summary: Migration to NumPy documentation style in ML (pyspark.ml.*) Key: SPARK-33251 URL: https://issues.apache.org/jira/browse/SPARK-33251 Project: Spark

[jira] [Created] (SPARK-33255) Use new API to construct ParquetFileReader and read Parquet footer

2020-10-26 Thread Yang Jie (Jira)
Yang Jie created SPARK-33255: Summary: Use new API to construct ParquetFileReader and read Parquet footer Key: SPARK-33255 URL: https://issues.apache.org/jira/browse/SPARK-33255 Project: Spark

[jira] [Updated] (SPARK-33249) Add status plugin for live application

2020-10-26 Thread Weiyi Kong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiyi Kong updated SPARK-33249: --- Description: There are cases that developer may want to extend the current REST API of Web UI. In

[jira] [Updated] (SPARK-33249) Add status plugin for live application

2020-10-26 Thread Weiyi Kong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiyi Kong updated SPARK-33249: --- Description: There are cases that developer may want to extend the current REST API of Web UI. In

[jira] [Commented] (SPARK-33248) Add a configuration to control the legacy behavior of whether need to pad null value when value size less then schema size

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221087#comment-17221087 ] Apache Spark commented on SPARK-33248: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-33248) Add a configuration to control the legacy behavior of whether need to pad null value when value size less then schema size

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33248: Assignee: Apache Spark > Add a configuration to control the legacy behavior of whether

[jira] [Commented] (SPARK-33248) Add a configuration to control the legacy behavior of whether need to pad null value when value size less then schema size

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221084#comment-17221084 ] Apache Spark commented on SPARK-33248: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-33248) Add a configuration to control the legacy behavior of whether need to pad null value when value size less then schema size

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33248: Assignee: (was: Apache Spark) > Add a configuration to control the legacy behavior

[jira] [Updated] (SPARK-32085) Migrate to NumPy documentation style

2020-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32085: - Affects Version/s: (was: 3.0.0) 3.1.0 > Migrate to NumPy

[jira] [Created] (SPARK-33249) Add status plugin for live application

2020-10-26 Thread Weiyi Kong (Jira)
Weiyi Kong created SPARK-33249: -- Summary: Add status plugin for live application Key: SPARK-33249 URL: https://issues.apache.org/jira/browse/SPARK-33249 Project: Spark Issue Type: New Feature

[jira] [Assigned] (SPARK-32084) Replace dictionary-based function definitions to proper functions in functions.py

2020-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32084: Assignee: Maciej Szymkiewicz > Replace dictionary-based function definitions to proper

[jira] [Resolved] (SPARK-32084) Replace dictionary-based function definitions to proper functions in functions.py

2020-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32084. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30143

[jira] [Updated] (SPARK-33248) Add a configuration to control the legacy behavior of whether need to pad null value when value size less then schema size

2020-10-26 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-33248: -- Description: Add a configuration to control the legacy behavior of whether need to pad null value

[jira] [Resolved] (SPARK-33238) Add a configuration to control the legacy behavior of whether need to pad null value when value size less then schema size

2020-10-26 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-33238. --- Resolution: Duplicate > Add a configuration to control the legacy behavior of whether need to pad

[jira] [Created] (SPARK-33248) Add a configuration to control the legacy behavior of whether need to pad null value when value size less then schema size

2020-10-26 Thread angerszhu (Jira)
angerszhu created SPARK-33248: - Summary: Add a configuration to control the legacy behavior of whether need to pad null value when value size less then schema size Key: SPARK-33248 URL:

[jira] [Created] (SPARK-33247) Improve examples and scenarios in docstrings

2020-10-26 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-33247: Summary: Improve examples and scenarios in docstrings Key: SPARK-33247 URL: https://issues.apache.org/jira/browse/SPARK-33247 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-32388) TRANSFORM when schema less should keep same with hive

2020-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32388. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29421

[jira] [Commented] (SPARK-32185) User Guide - Monitoring

2020-10-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221047#comment-17221047 ] Hyukjin Kwon commented on SPARK-32185: -- Thanks! > User Guide - Monitoring >

[jira] [Commented] (SPARK-32789) Wildcards not working in get_json_object

2020-10-26 Thread Aoyuan Liao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221044#comment-17221044 ] Aoyuan Liao commented on SPARK-32789: - [~tuhren] Not sure if HIve supports wildcard for dictionary.

[jira] [Resolved] (SPARK-32789) Wildcards not working in get_json_object

2020-10-26 Thread Aoyuan Liao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aoyuan Liao resolved SPARK-32789. - Resolution: Not A Problem > Wildcards not working in get_json_object >

[jira] [Assigned] (SPARK-33231) Make podCreationTimeout configurable

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33231: Assignee: Apache Spark > Make podCreationTimeout configurable >

[jira] [Commented] (SPARK-33231) Make podCreationTimeout configurable

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221031#comment-17221031 ] Apache Spark commented on SPARK-33231: -- User 'holdenk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33231) Make podCreationTimeout configurable

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33231: Assignee: (was: Apache Spark) > Make podCreationTimeout configurable >

[jira] [Commented] (SPARK-33231) Make podCreationTimeout configurable

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221032#comment-17221032 ] Apache Spark commented on SPARK-33231: -- User 'holdenk' has created a pull request for this issue:

[jira] [Commented] (SPARK-32405) Apply table options while creating tables in JDBC Table Catalog

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221029#comment-17221029 ] Apache Spark commented on SPARK-32405: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32405) Apply table options while creating tables in JDBC Table Catalog

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32405: Assignee: Apache Spark > Apply table options while creating tables in JDBC Table Catalog

[jira] [Commented] (SPARK-32405) Apply table options while creating tables in JDBC Table Catalog

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221028#comment-17221028 ] Apache Spark commented on SPARK-32405: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32405) Apply table options while creating tables in JDBC Table Catalog

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32405: Assignee: (was: Apache Spark) > Apply table options while creating tables in JDBC

[jira] [Assigned] (SPARK-33237) Use default Hadoop profile by removing explicit `-Phadoop-2.7` from K8s IT Jenkins job

2020-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33237: - Assignee: Dongjoon Hyun > Use default Hadoop profile by removing explicit

[jira] [Resolved] (SPARK-33237) Use default Hadoop profile by removing explicit `-Phadoop-2.7` from K8s IT Jenkins job

2020-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33237. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30153

[jira] [Commented] (SPARK-33150) Groupby key may not be unique when using window

2020-10-26 Thread Aoyuan Liao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220984#comment-17220984 ] Aoyuan Liao commented on SPARK-33150: - [~DieterDP] After I looked deeper into code, the issue is not

[jira] [Updated] (SPARK-33228) Don't uncache data when replacing an existing view having the same plan

2020-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33228: -- Fix Version/s: (was: 2.4.8) > Don't uncache data when replacing an existing view having

[jira] [Resolved] (SPARK-33230) FileOutputWriter jobs have duplicate JobIDs if launched in same second

2020-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33230. --- Fix Version/s: 2.4.8 3.0.2 3.1.0 Resolution:

[jira] [Assigned] (SPARK-33230) FileOutputWriter jobs have duplicate JobIDs if launched in same second

2020-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33230: - Assignee: Steve Loughran > FileOutputWriter jobs have duplicate JobIDs if launched in

[jira] [Updated] (SPARK-33237) Use default Hadoop profile by removing explicit `-Phadoop-2.7` from K8s IT Jenkins job

2020-10-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33237: -- Component/s: Kubernetes > Use default Hadoop profile by removing explicit `-Phadoop-2.7` from

[jira] [Assigned] (SPARK-33237) Use default Hadoop profile by removing explicit `-Phadoop-2.7` from K8s IT Jenkins job

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33237: Assignee: Apache Spark > Use default Hadoop profile by removing explicit `-Phadoop-2.7`

[jira] [Commented] (SPARK-33237) Use default Hadoop profile by removing explicit `-Phadoop-2.7` from K8s IT Jenkins job

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220943#comment-17220943 ] Apache Spark commented on SPARK-33237: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-33237) Use default Hadoop profile by removing explicit `-Phadoop-2.7` from K8s IT Jenkins job

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33237: Assignee: (was: Apache Spark) > Use default Hadoop profile by removing explicit

[jira] [Commented] (SPARK-32185) User Guide - Monitoring

2020-10-26 Thread Abhijeet Prasad (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220902#comment-17220902 ] Abhijeet Prasad commented on SPARK-32185: - Hey, sorry for not updating this issue. I have been

[jira] [Closed] (SPARK-33197) Changes to spark.sql.analyzer.maxIterations do not take effect at runtime

2020-10-26 Thread Yuning Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuning Zhang closed SPARK-33197. > Changes to spark.sql.analyzer.maxIterations do not take effect at runtime >

[jira] [Commented] (SPARK-19335) Spark should support doing an efficient DataFrame Upsert via JDBC

2020-10-26 Thread Denise Mauldin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220856#comment-17220856 ] Denise Mauldin commented on SPARK-19335: [~kevinyu98] Using AWS Glue to copy/update data between

[jira] [Commented] (SPARK-19335) Spark should support doing an efficient DataFrame Upsert via JDBC

2020-10-26 Thread Denise Mauldin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220855#comment-17220855 ] Denise Mauldin commented on SPARK-19335: +1 This is a major deficiency for using Spark in ETL

[jira] [Updated] (SPARK-33246) Spark SQL null semantics documentation is incorrect

2020-10-26 Thread Stuart White (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stuart White updated SPARK-33246: - Attachment: null-semantics.patch > Spark SQL null semantics documentation is incorrect >

[jira] [Created] (SPARK-33246) Spark SQL null semantics documentation is incorrect

2020-10-26 Thread Stuart White (Jira)
Stuart White created SPARK-33246: Summary: Spark SQL null semantics documentation is incorrect Key: SPARK-33246 URL: https://issues.apache.org/jira/browse/SPARK-33246 Project: Spark Issue

[jira] [Commented] (SPARK-23539) Add support for Kafka headers in Structured Streaming

2020-10-26 Thread Calvin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220813#comment-17220813 ] Calvin commented on SPARK-23539: [~dongjin]/[~kabhwan] Apologies for reviving this long-closed ticket,

[jira] [Commented] (SPARK-33228) Don't uncache data when replacing an existing view having the same plan

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220807#comment-17220807 ] Apache Spark commented on SPARK-33228: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-26043) Make SparkHadoopUtil private to Spark

2020-10-26 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220752#comment-17220752 ] Sean R. Owen commented on SPARK-26043: -- I don't have a strong opinion on it. [~vanzin] says it

[jira] [Commented] (SPARK-26043) Make SparkHadoopUtil private to Spark

2020-10-26 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220748#comment-17220748 ] Wenchen Fan commented on SPARK-26043: - A quick way is to copy-paste the code to your repo so that it

[jira] [Updated] (SPARK-33233) CUBE/ROLLUP can't support UnresolvedOrdinal

2020-10-26 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-33233: -- Description: Now spark support group by ordinal, but cube/rollup/groupingsets not support this. This

[jira] [Created] (SPARK-33245) Add built-in UDF - GETBIT

2020-10-26 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33245: --- Summary: Add built-in UDF - GETBIT Key: SPARK-33245 URL: https://issues.apache.org/jira/browse/SPARK-33245 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-33183) Bug in optimizer rule EliminateSorts

2020-10-26 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-33183: - Affects Version/s: (was: 3.0.1) (was: 3.0.0)

[jira] [Resolved] (SPARK-33204) `Event Timeline` in Spark Job UI sometimes cannot be opened

2020-10-26 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-33204. Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30119

[jira] [Commented] (SPARK-33233) CUBE/ROLLUP can't support UnresolvedOrdinal

2020-10-26 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220648#comment-17220648 ] Takeshi Yamamuro commented on SPARK-33233: -- Please fill the description. > CUBE/ROLLUP can't

[jira] [Updated] (SPARK-33233) CUBE/ROLLUP can't support UnresolvedOrdinal

2020-10-26 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-33233: - Issue Type: Improvement (was: Bug) > CUBE/ROLLUP can't support UnresolvedOrdinal >

[jira] [Commented] (SPARK-33223) Expose state information on SS UI

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220637#comment-17220637 ] Apache Spark commented on SPARK-33223: -- User 'gaborgsomogyi' has created a pull request for this

[jira] [Assigned] (SPARK-33223) Expose state information on SS UI

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33223: Assignee: Apache Spark > Expose state information on SS UI >

[jira] [Assigned] (SPARK-33223) Expose state information on SS UI

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33223: Assignee: (was: Apache Spark) > Expose state information on SS UI >

[jira] [Resolved] (SPARK-33075) Only disable auto bucketed scan for cached query

2020-10-26 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-33075. -- Fix Version/s: 3.1.0 Assignee: Cheng Su Resolution: Fixed Resolved by 

[jira] [Commented] (SPARK-32188) API Reference

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220625#comment-17220625 ] Apache Spark commented on SPARK-32188: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-32188) API Reference

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220624#comment-17220624 ] Apache Spark commented on SPARK-32188: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-33243) Add numpydoc into documentation dependency

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33243: Assignee: Apache Spark > Add numpydoc into documentation dependency >

[jira] [Commented] (SPARK-33243) Add numpydoc into documentation dependency

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220617#comment-17220617 ] Apache Spark commented on SPARK-33243: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-33243) Add numpydoc into documentation dependency

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33243: Assignee: (was: Apache Spark) > Add numpydoc into documentation dependency >

[jira] [Assigned] (SPARK-33244) Unify the code paths for spark.table and spark.read.table

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33244: Assignee: (was: Apache Spark) > Unify the code paths for spark.table and

[jira] [Assigned] (SPARK-33244) Unify the code paths for spark.table and spark.read.table

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33244: Assignee: Apache Spark > Unify the code paths for spark.table and spark.read.table >

[jira] [Commented] (SPARK-33244) Unify the code paths for spark.table and spark.read.table

2020-10-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220615#comment-17220615 ] Apache Spark commented on SPARK-33244: -- User 'xuanyuanking' has created a pull request for this

[jira] [Created] (SPARK-33244) Unify the code paths for spark.table and spark.read.table

2020-10-26 Thread Yuanjian Li (Jira)
Yuanjian Li created SPARK-33244: --- Summary: Unify the code paths for spark.table and spark.read.table Key: SPARK-33244 URL: https://issues.apache.org/jira/browse/SPARK-33244 Project: Spark

  1   2   >