[jira] [Commented] (SPARK-32798) Make unionByName optionally fill missing columns with nulls in PySpark

2020-09-06 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191434#comment-17191434 ] Haejoon Lee commented on SPARK-32798: - I'm interested in this. I'll take this one! > Make

[jira] [Updated] (SPARK-32812) Run tests script for Python fails in certain environments

2020-09-07 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-32812: Summary: Run tests script for Python fails in certain environments (was: Run tests script for

[jira] [Commented] (SPARK-32189) Development - Setting up PyCharm

2020-09-03 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189868#comment-17189868 ] Haejoon Lee commented on SPARK-32189: - I am working on this > Development - Setting up PyCharm >

[jira] [Created] (SPARK-32812) Run tests script for Python fails on local environment

2020-09-07 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-32812: --- Summary: Run tests script for Python fails on local environment Key: SPARK-32812 URL: https://issues.apache.org/jira/browse/SPARK-32812 Project: Spark Issue

[jira] [Commented] (SPARK-33253) Migration to NumPy documentation style in Streaming (pyspark.streaming.*)

2020-10-29 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222919#comment-17222919 ] Haejoon Lee commented on SPARK-33253: - I'll take this one > Migration to NumPy documentation style

[jira] [Updated] (SPARK-34190) Supplement the description for Python Package Management

2021-01-21 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-34190: Summary: Supplement the description for Python Package Management (was: Supplement the

[jira] [Updated] (SPARK-34190) Supplement the description in the document

2021-01-21 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-34190: Description: There is lack of explanation for "Using Virtualenv" chapter. It says "It packs the

[jira] [Created] (SPARK-34190) Fix inconsistent docs in Python Package Management

2021-01-21 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-34190: --- Summary: Fix inconsistent docs in Python Package Management Key: SPARK-34190 URL: https://issues.apache.org/jira/browse/SPARK-34190 Project: Spark Issue Type:

[jira] [Updated] (SPARK-34190) Supplement the description in the document

2021-01-21 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-34190: Summary: Supplement the description in the document (was: Fix inconsistent docs in Python

[jira] [Updated] (SPARK-34190) Supplement the description in the document

2021-01-21 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-34190: Description: There is lack of explanation for "Using Virtualenv" chapter. It says "It packs the

[jira] [Updated] (SPARK-34190) Supplement the description in the document

2021-01-21 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-34190: Description: There is lack of explanation for "Using Virtualenv" chapter. It says "It packs the

[jira] [Updated] (SPARK-34190) Supplement the description in the document

2021-01-21 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-34190: Description: There is lack of explanation for "Using Virtualenv" chapter. It says "It packs the

[jira] [Commented] (SPARK-35696) Refine the code examples in the pandas APIs on Spark documentation.

2021-06-14 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17363377#comment-17363377 ] Haejoon Lee commented on SPARK-35696: - I'm working on this > Refine the code examples in the pandas

[jira] [Updated] (SPARK-35696) Refine the code examples in the pandas API on Spark documentation.

2021-06-14 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35696: Summary: Refine the code examples in the pandas API on Spark documentation. (was: Refine the

[jira] [Created] (SPARK-35696) Refine the code examples in the pandas APIs on Spark documentation.

2021-06-09 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35696: --- Summary: Refine the code examples in the pandas APIs on Spark documentation. Key: SPARK-35696 URL: https://issues.apache.org/jira/browse/SPARK-35696 Project: Spark

[jira] [Updated] (SPARK-35591) Rename "Koalas" to "pandas APIs on Spark" in the documents

2021-06-13 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35591: Summary: Rename "Koalas" to "pandas APIs on Spark" in the documents (was: Rename "Koalas" to

[jira] [Created] (SPARK-35750) Rename "pandas APIs on Spark" to "pandas API on Spark" in the documents

2021-06-13 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35750: --- Summary: Rename "pandas APIs on Spark" to "pandas API on Spark" in the documents Key: SPARK-35750 URL: https://issues.apache.org/jira/browse/SPARK-35750 Project: Spark

[jira] [Created] (SPARK-35805) Pandas API on Spark improvements

2021-06-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35805: --- Summary: Pandas API on Spark improvements Key: SPARK-35805 URL: https://issues.apache.org/jira/browse/SPARK-35805 Project: Spark Issue Type: Umbrella

[jira] [Created] (SPARK-35807) Rename the `num_files` argument

2021-06-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35807: --- Summary: Rename the `num_files` argument Key: SPARK-35807 URL: https://issues.apache.org/jira/browse/SPARK-35807 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-35811) Deprecate DataFrame.to_spark_io

2021-06-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35811: --- Summary: Deprecate DataFrame.to_spark_io Key: SPARK-35811 URL: https://issues.apache.org/jira/browse/SPARK-35811 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-35810) Remove ps.broadcast API

2021-06-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35810: --- Summary: Remove ps.broadcast API Key: SPARK-35810 URL: https://issues.apache.org/jira/browse/SPARK-35810 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-35806) Rename the `mode` argument to avoid confusion with `mode` argument in pandas

2021-06-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35806: --- Summary: Rename the `mode` argument to avoid confusion with `mode` argument in pandas Key: SPARK-35806 URL: https://issues.apache.org/jira/browse/SPARK-35806 Project:

[jira] [Created] (SPARK-35812) Throw an error if `version` and `timestamp` are used together in DataFrame.to_delta.

2021-06-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35812: --- Summary: Throw an error if `version` and `timestamp` are used together in DataFrame.to_delta. Key: SPARK-35812 URL: https://issues.apache.org/jira/browse/SPARK-35812

[jira] [Created] (SPARK-35809) Add `index_col` argument for ps.sql.

2021-06-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35809: --- Summary: Add `index_col` argument for ps.sql. Key: SPARK-35809 URL: https://issues.apache.org/jira/browse/SPARK-35809 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-35808) Always enable the `pandas_metadata` in DataFrame.parquet

2021-06-17 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35808: --- Summary: Always enable the `pandas_metadata` in DataFrame.parquet Key: SPARK-35808 URL: https://issues.apache.org/jira/browse/SPARK-35808 Project: Spark Issue

[jira] [Updated] (SPARK-35873) Cleanup the version logic from the pandas API on Spark

2021-06-24 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35873: Summary: Cleanup the version logic from the pandas API on Spark (was: Remove the version check

[jira] [Updated] (SPARK-35807) Rename the `num_files` argument

2021-06-17 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35807: Description: We should rename the num_files argument in [DataFrame.to_csv 

[jira] [Resolved] (SPARK-35807) Rename the `num_files` argument

2021-06-18 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-35807. - Resolution: Invalid Seems like we need more discussion. Let me just close this for now. >

[jira] [Resolved] (SPARK-35806) Rename the `mode` argument to avoid confusion with `mode` argument in pandas

2021-06-18 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-35806. - Resolution: Invalid Seems like we need more discussion. Let me just close this for now. >

[jira] [Created] (SPARK-35364) Renaming the existing Koalas related codes.

2021-05-10 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35364: --- Summary: Renaming the existing Koalas related codes. Key: SPARK-35364 URL: https://issues.apache.org/jira/browse/SPARK-35364 Project: Spark Issue Type:

[jira] [Commented] (SPARK-35364) Renaming the existing Koalas related codes.

2021-05-10 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342262#comment-17342262 ] Haejoon Lee commented on SPARK-35364: - I'm working on this > Renaming the existing Koalas related

[jira] [Updated] (SPARK-35433) Move CSV data source options from Python and Scala into a single page.

2021-05-18 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35433: Description: Refer to https://issues.apache.org/jira/browse/SPARK-34491 > Move CSV data source

[jira] [Created] (SPARK-35433) Move CSV data source options from Python and Scala into a single page.

2021-05-18 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35433: --- Summary: Move CSV data source options from Python and Scala into a single page. Key: SPARK-35433 URL: https://issues.apache.org/jira/browse/SPARK-35433 Project: Spark

[jira] [Updated] (SPARK-35395) Move ORC data source options from Python and Scala into a single page

2021-05-12 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35395: Summary: Move ORC data source options from Python and Scala into a single page (was: Move ORC

[jira] [Updated] (SPARK-35395) Move ORC data source options from Python and Scala into a single pa

2021-05-12 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35395: Description: Refer to https://issues.apache.org/jira/browse/SPARK-34491 > Move ORC data source

[jira] [Created] (SPARK-35395) Move ORC data source options from Python and Scala into a single pa

2021-05-12 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35395: --- Summary: Move ORC data source options from Python and Scala into a single pa Key: SPARK-35395 URL: https://issues.apache.org/jira/browse/SPARK-35395 Project: Spark

[jira] [Created] (SPARK-35481) Create more robust link for Data Source Options

2021-05-20 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35481: --- Summary: Create more robust link for Data Source Options Key: SPARK-35481 URL: https://issues.apache.org/jira/browse/SPARK-35481 Project: Spark Issue Type:

[jira] [Updated] (SPARK-35481) Create more robust link for Data Source Options

2021-05-20 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35481: Description: Now the link for the Data Source Options using /latest/, but it possibly be broken

[jira] [Updated] (SPARK-35481) Create more robust link for Data Source Options

2021-05-20 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35481: Description: Now the link for the Data Source Options using /latest/, but it possibly be broken

[jira] [Created] (SPARK-35453) Move Koalas accessor to pandas_on_spark accessor

2021-05-19 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35453: --- Summary: Move Koalas accessor to pandas_on_spark accessor Key: SPARK-35453 URL: https://issues.apache.org/jira/browse/SPARK-35453 Project: Spark Issue Type:

[jira] [Commented] (SPARK-35453) Move Koalas accessor to pandas_on_spark accessor

2021-05-19 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348021#comment-17348021 ] Haejoon Lee commented on SPARK-35453: - I'm working on this > Move Koalas accessor to

[jira] [Created] (SPARK-35528) Add more methods for Data Source Options pages

2021-05-26 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35528: --- Summary: Add more methods for Data Source Options pages Key: SPARK-35528 URL: https://issues.apache.org/jira/browse/SPARK-35528 Project: Spark Issue Type:

[jira] [Updated] (SPARK-35499) Apply black to pandas API on Spark codes.

2021-05-24 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35499: Parent: SPARK-34849 Issue Type: Sub-task (was: Improvement) > Apply black to pandas API

[jira] [Commented] (SPARK-35433) Move CSV data source options from Python and Scala into a single page.

2021-05-24 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17350771#comment-17350771 ] Haejoon Lee commented on SPARK-35433: - I'm working on this > Move CSV data source options from

[jira] [Created] (SPARK-35509) Move TEXT data source options from Python and Scala into a single page.

2021-05-25 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35509: --- Summary: Move TEXT data source options from Python and Scala into a single page. Key: SPARK-35509 URL: https://issues.apache.org/jira/browse/SPARK-35509 Project:

[jira] [Created] (SPARK-35603) Move data source options from R into a single page.

2021-06-01 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35603: --- Summary: Move data source options from R into a single page. Key: SPARK-35603 URL: https://issues.apache.org/jira/browse/SPARK-35603 Project: Spark Issue

[jira] [Commented] (SPARK-35605) Move pandas_on_spark accessor to the Spark DataFrame.

2021-06-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355438#comment-17355438 ] Haejoon Lee commented on SPARK-35605: - I'm working on this > Move pandas_on_spark accessor to the

[jira] [Created] (SPARK-35605) Move pandas_on_spark accessor to the Spark DataFrame.

2021-06-01 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35605: --- Summary: Move pandas_on_spark accessor to the Spark DataFrame. Key: SPARK-35605 URL: https://issues.apache.org/jira/browse/SPARK-35605 Project: Spark Issue

[jira] [Updated] (SPARK-35605) Move pandas_on_spark accessor to the Spark DataFrame.

2021-06-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35605: Description: Inspired by https://github.com/apache/spark/pull/32729#discussion_r643591322, As

[jira] [Updated] (SPARK-35603) Move data source options from R into a single page.

2021-06-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35603: Description: We should consolidate the data source options from R documentation as well like we

[jira] [Updated] (SPARK-35603) Move data source options from R into a single page.

2021-06-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35603: Parent: (was: SPARK-34491) Issue Type: Documentation (was: Sub-task) > Move data

[jira] [Created] (SPARK-35583) Move JDBC data source options from Python and Scala into a single page

2021-05-31 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35583: --- Summary: Move JDBC data source options from Python and Scala into a single page Key: SPARK-35583 URL: https://issues.apache.org/jira/browse/SPARK-35583 Project: Spark

[jira] [Commented] (SPARK-35582) Remove # noqa in Python API documents.

2021-05-31 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17354732#comment-17354732 ] Haejoon Lee commented on SPARK-35582: - I'm working on this > Remove # noqa in Python API documents.

[jira] [Created] (SPARK-35582) Remove # noqa in Python API documents.

2021-05-31 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35582: --- Summary: Remove # noqa in Python API documents. Key: SPARK-35582 URL: https://issues.apache.org/jira/browse/SPARK-35582 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-35539) Restore to_koalas to keep the backward compatibility

2021-06-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35539: Description: In https://github.com/apache/spark/pull/32516 we removed the to_koalas but we'd

[jira] [Updated] (SPARK-35539) Restore to_koalas to keep the backward compatibility

2021-06-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35539: Description: In

[jira] [Updated] (SPARK-35539) Restore to_koalas to keep the backward compatibility

2021-06-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35539: Description: In

[jira] [Created] (SPARK-35591) Rename Koalas to pandas API on Spark

2021-06-01 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35591: --- Summary: Rename Koalas to pandas API on Spark Key: SPARK-35591 URL: https://issues.apache.org/jira/browse/SPARK-35591 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-35591) Rename Koalas to pandas API on Spark

2021-06-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17354854#comment-17354854 ] Haejoon Lee commented on SPARK-35591: - I'm working on this > Rename Koalas to pandas API on Spark >

[jira] [Updated] (SPARK-35591) Rename Koalas to pandas API on Spark in the documents

2021-06-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35591: Summary: Rename Koalas to pandas API on Spark in the documents (was: Rename Koalas to pandas API

[jira] [Updated] (SPARK-35591) Rename "Koalas" to "pandas API on Spark" in the documents

2021-06-01 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35591: Summary: Rename "Koalas" to "pandas API on Spark" in the documents (was: Rename Koalas to pandas

[jira] [Created] (SPARK-35499) Apply black to pandas API on Spark codes.

2021-05-24 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35499: --- Summary: Apply black to pandas API on Spark codes. Key: SPARK-35499 URL: https://issues.apache.org/jira/browse/SPARK-35499 Project: Spark Issue Type:

[jira] [Updated] (SPARK-35499) Apply black to pandas API on Spark codes.

2021-05-24 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35499: Description: Make it easier and more efficient to static analysis, we'd better to apply `black`

[jira] [Updated] (SPARK-35603) Add data source options link for R API documentation.

2021-06-06 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35603: Description: We should add the data source options link for R documentation as well like we did

[jira] [Updated] (SPARK-35603) Add data source options link for R API documentation.

2021-06-06 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35603: Summary: Add data source options link for R API documentation. (was: Move data source options

[jira] [Commented] (SPARK-35591) Rename "Koalas" to "pandas API on Spark" in the documents

2021-06-07 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17358538#comment-17358538 ] Haejoon Lee commented on SPARK-35591: - [~pingsutw] Thanks for the concern, but I almost done with

[jira] [Commented] (SPARK-35539) Restore to_koalas to keep the backward compatibility

2021-05-26 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17352250#comment-17352250 ] Haejoon Lee commented on SPARK-35539: - I'm working on this > Restore to_koalas to keep the backward

[jira] [Created] (SPARK-35539) Restore to_koalas to keep the backward compatibility

2021-05-26 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35539: --- Summary: Restore to_koalas to keep the backward compatibility Key: SPARK-35539 URL: https://issues.apache.org/jira/browse/SPARK-35539 Project: Spark Issue

[jira] [Updated] (SPARK-35528) Add more options at Data Source Options pages

2021-05-26 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35528: Summary: Add more options at Data Source Options pages (was: Add CREATE TABLE with OPTIONS at

[jira] [Created] (SPARK-35523) Fix the default value properly in Data Source Options page

2021-05-25 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35523: --- Summary: Fix the default value properly in Data Source Options page Key: SPARK-35523 URL: https://issues.apache.org/jira/browse/SPARK-35523 Project: Spark

[jira] [Created] (SPARK-35569) Document the deprecation of Koalas Accessor after porting the documentation.

2021-05-30 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35569: --- Summary: Document the deprecation of Koalas Accessor after porting the documentation. Key: SPARK-35569 URL: https://issues.apache.org/jira/browse/SPARK-35569 Project:

[jira] [Commented] (SPARK-35499) Apply black to pandas API on Spark codes.

2021-06-03 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17357045#comment-17357045 ] Haejoon Lee commented on SPARK-35499: - I'm working on this > Apply black to pandas API on Spark

[jira] [Updated] (SPARK-35605) Move to_pandas_on_spark to the Spark DataFrame.

2021-06-23 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35605: Summary: Move to_pandas_on_spark to the Spark DataFrame. (was: Move pandas_on_spark accessor to

[jira] [Created] (SPARK-35873) Remove the version check from the __init__ from pyspark.pandas

2021-06-24 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35873: --- Summary: Remove the version check from the __init__ from pyspark.pandas Key: SPARK-35873 URL: https://issues.apache.org/jira/browse/SPARK-35873 Project: Spark

[jira] [Commented] (SPARK-35929) Schema inference of nested structs defaults to map

2021-06-29 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17371199#comment-17371199 ] Haejoon Lee commented on SPARK-35929: - I'm working on this > Schema inference of nested structs

[jira] [Updated] (SPARK-35929) Schema inference of nested structs defaults to map

2021-06-29 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35929: Description: Inferring schema for struct columns causing schema issues as below. {code:java} >>>

[jira] [Updated] (SPARK-35929) Schema inference of nested structs defaults to map

2021-06-29 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-35929: Description: Inferring schema for struct columns causing schema issues as below. {code:java} data

[jira] [Created] (SPARK-35929) Schema inference of nested structs defaults to map

2021-06-29 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-35929: --- Summary: Schema inference of nested structs defaults to map Key: SPARK-35929 URL: https://issues.apache.org/jira/browse/SPARK-35929 Project: Spark Issue Type:

[jira] [Created] (SPARK-34849) SPIP: Support pandas API layer on PySpark

2021-03-24 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-34849: --- Summary: SPIP: Support pandas API layer on PySpark Key: SPARK-34849 URL: https://issues.apache.org/jira/browse/SPARK-34849 Project: Spark Issue Type: Umbrella

[jira] [Updated] (SPARK-34849) SPIP: Support pandas API layer on PySpark

2021-03-26 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-34849: Description: This is a SPIP for porting [Koalas project|https://github.com/databricks/koalas] to

[jira] [Updated] (SPARK-34849) SPIP: Support pandas API layer on PySpark

2021-03-26 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-34849: Description: This is a SPIP for porting [Koalas project|https://github.com/databricks/koalas] to

[jira] [Updated] (SPARK-34849) SPIP: Support pandas API layer on PySpark

2021-03-26 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-34849: Description: This is a SPIP for porting [Koalas project|https://github.com/databricks/koalas] to

[jira] [Comment Edited] (SPARK-34849) SPIP: Support pandas API layer on PySpark

2021-03-26 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309492#comment-17309492 ] Haejoon Lee edited comment on SPARK-34849 at 3/26/21, 3:23 PM: --- The

[jira] [Commented] (SPARK-34849) SPIP: Support pandas API layer on PySpark

2021-03-26 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309492#comment-17309492 ] Haejoon Lee commented on SPARK-34849: - The SPIP voting started at

[jira] [Updated] (SPARK-34849) SPIP: Support pandas API layer on PySpark

2021-03-26 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-34849: Description: This is a SPIP for porting [Koalas project|https://github.com/databricks/koalas] to

[jira] [Updated] (SPARK-34849) SPIP: Support pandas API layer on PySpark

2021-03-25 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-34849: Description: This is a SPIP for porting [Koalas project|https://github.com/databricks/koalas] to

[jira] [Created] (SPARK-34886) Integrate Koalas tests into PySpark

2021-03-28 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-34886: --- Summary: Integrate Koalas tests into PySpark Key: SPARK-34886 URL: https://issues.apache.org/jira/browse/SPARK-34886 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-34890) Integrate Koalas main API code into PySpark

2021-03-28 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-34890: --- Summary: Integrate Koalas main API code into PySpark Key: SPARK-34890 URL: https://issues.apache.org/jira/browse/SPARK-34890 Project: Spark Issue Type:

[jira] [Created] (SPARK-34887) Integrate Koala dependencies into PySpark

2021-03-28 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-34887: --- Summary: Integrate Koala dependencies into PySpark Key: SPARK-34887 URL: https://issues.apache.org/jira/browse/SPARK-34887 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-34885) Integrate Koalas documentation into PySpark

2021-03-28 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-34885: --- Summary: Integrate Koalas documentation into PySpark Key: SPARK-34885 URL: https://issues.apache.org/jira/browse/SPARK-34885 Project: Spark Issue Type:

[jira] [Created] (SPARK-34941) Enable mypy test for pyspark.pandas

2021-04-02 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-34941: --- Summary: Enable mypy test for pyspark.pandas Key: SPARK-34941 URL: https://issues.apache.org/jira/browse/SPARK-34941 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-34941) Enable mypy for pandas-on-Spark

2021-04-02 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-34941: Summary: Enable mypy for pandas-on-Spark (was: Enable mypy test for pyspark.pandas) > Enable

[jira] [Created] (SPARK-34943) Upgrade flake8 to 3.8.0 or above in Jenkins

2021-04-02 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-34943: --- Summary: Upgrade flake8 to 3.8.0 or above in Jenkins Key: SPARK-34943 URL: https://issues.apache.org/jira/browse/SPARK-34943 Project: Spark Issue Type:

[jira] [Updated] (SPARK-34943) Upgrade flake8 to 3.8.0 or above in Jenkins

2021-04-02 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-34943: Description: We should upgrade the flake8 installed in Jenkins because the flake8 test for

[jira] [Updated] (SPARK-34943) Upgrade flake8 to 3.8.0 or above in Jenkins

2021-04-03 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-34943: Description: In flake8 < 3.8.0, F401 error occurs for imports in if statements when

[jira] [Updated] (SPARK-34943) Upgrade flake8 to 3.8.0 or above in Jenkins

2021-04-03 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-34943: Description: In flake8 < 3.8.0, F401 error occurs for imports in "if" statements when

[jira] [Updated] (SPARK-34943) Upgrade flake8 to 3.8.0 or above in Jenkins

2021-04-03 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-34943: Description: In flake8 < 3.8.0, F401 error occurs for imports in *if* statements when

[jira] [Commented] (SPARK-34943) Upgrade flake8 to 3.8.0 or above in Jenkins

2021-04-03 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17314405#comment-17314405 ] Haejoon Lee commented on SPARK-34943: - [~hyukjin.kwon] Updated. Thanks! > Upgrade flake8 to 3.8.0

[jira] [Updated] (SPARK-34491) Move data source options docs for Python and Scala into a single page

2021-03-10 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-34491: Description: Currently we documents options of various data sources for Python and Scala in a

[jira] [Commented] (SPARK-34493) Create "TEXT Files" page for Data Source documents.

2021-03-13 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17301034#comment-17301034 ] Haejoon Lee commented on SPARK-34493: - I'm working on this > Create "TEXT Files" page for Data

  1   2   3   4   5   6   7   8   9   10   >