[jira] [Commented] (SPARK-30037) Customize krb5.conf to test Kafka delegation token with MiniKDC

2020-06-25 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146008#comment-17146008 ] angerszhu commented on SPARK-30037: --- [~gsomogyi]  No, that pr solve jdk11 problem by change krb5.conf

[jira] [Commented] (SPARK-26968) option("quoteMode", "NON_NUMERIC") have no effect on a CSV generation

2020-06-25 Thread Jijo Sunny (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146003#comment-17146003 ] Jijo Sunny commented on SPARK-26968: The external spark-csv library is only intended for Spark 1.x

[jira] [Commented] (SPARK-20624) SPIP: Add better handling for node shutdown

2020-06-25 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17145978#comment-17145978 ] Holden Karau commented on SPARK-20624: -- The SPIP document is at  

[jira] [Commented] (SPARK-20624) SPIP: Add better handling for node shutdown

2020-06-25 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17145977#comment-17145977 ] Holden Karau commented on SPARK-20624: -- The current design document is at 

[jira] [Commented] (SPARK-32094) Patch cloudpickle.py with typing module side-effect fix

2020-06-25 Thread Suzen Fylke (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17145968#comment-17145968 ] Suzen Fylke commented on SPARK-32094: - Yes! Will do. > Patch cloudpickle.py with typing module

[jira] [Updated] (SPARK-32096) Support top-N sort for Spark SQL rank window function

2020-06-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32096: - Fix Version/s: (was: 3.1.0) > Support top-N sort for Spark SQL rank window function >

[jira] [Commented] (SPARK-32094) Patch cloudpickle.py with typing module side-effect fix

2020-06-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17145967#comment-17145967 ] Hyukjin Kwon commented on SPARK-32094: -- Are you interested in submitting a PR to upgrade? > Patch

[jira] [Updated] (SPARK-32096) Support top-N sort for Spark SQL rank window function

2020-06-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32096: - Target Version/s: (was: 3.1.0) > Support top-N sort for Spark SQL rank window function >

[jira] [Commented] (SPARK-32088) test of pyspark.sql.functions.timestamp_seconds failed if non-american timezone setting

2020-06-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17145966#comment-17145966 ] Hyukjin Kwon commented on SPARK-32088: -- [~huangtianhua] are you interested in a PR? I think you can

[jira] [Resolved] (SPARK-32025) CSV schema inference with boolean & integer

2020-06-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32025. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28896

[jira] [Assigned] (SPARK-32025) CSV schema inference with boolean & integer

2020-06-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32025: Assignee: Pablo Langa Blanco > CSV schema inference with boolean & integer >

[jira] [Updated] (SPARK-20624) SPIP: Add better handling for node shutdown

2020-06-25 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-20624: - Summary: SPIP: Add better handling for node shutdown (was: Add better handling for node

[jira] [Resolved] (SPARK-32100) Add WorkerDecommissionExtendedSuite

2020-06-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32100. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28929

[jira] [Assigned] (SPARK-32100) Add WorkerDecommissionExtendedSuite

2020-06-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32100: - Assignee: Dongjoon Hyun > Add WorkerDecommissionExtendedSuite >

[jira] [Commented] (SPARK-18381) Wrong date conversion between spark and python for dates before 1583

2020-06-25 Thread Tim Stavenger (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17145813#comment-17145813 ] Tim Stavenger commented on SPARK-18381: --- This date issue appears to be fixed (at least in my

[jira] [Issue Comment Deleted] (SPARK-32100) Add WorkerDecommissionExtendedSuite

2020-06-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32100: -- Comment: was deleted (was: User 'dongjoon-hyun' has created a pull request for this issue:

[jira] [Commented] (SPARK-32100) Add WorkerDecommissionExtendedSuite

2020-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17145803#comment-17145803 ] Apache Spark commented on SPARK-32100: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-32100) Add WorkerDecommissionExtendedSuite

2020-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32100: Assignee: (was: Apache Spark) > Add WorkerDecommissionExtendedSuite >

[jira] [Assigned] (SPARK-32100) Add WorkerDecommissionExtendedSuite

2020-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32100: Assignee: Apache Spark > Add WorkerDecommissionExtendedSuite >

[jira] [Commented] (SPARK-32100) Add WorkerDecommissionExtendedSuite

2020-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17145801#comment-17145801 ] Apache Spark commented on SPARK-32100: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Created] (SPARK-32100) Add WorkerDecommissionExtendedSuite

2020-06-25 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-32100: - Summary: Add WorkerDecommissionExtendedSuite Key: SPARK-32100 URL: https://issues.apache.org/jira/browse/SPARK-32100 Project: Spark Issue Type:

[jira] [Commented] (SPARK-28489) KafkaOffsetRangeCalculator.getRanges may drop offsets

2020-06-25 Thread Shixiong Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17145719#comment-17145719 ] Shixiong Zhu commented on SPARK-28489: -- [~klaustelenius] No. The "minPartitions" option for batch

[jira] [Assigned] (SPARK-32098) Use iloc for positional slicing instead of direct slicing in createDataFrame with Arrow

2020-06-25 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-32098: Assignee: Hyukjin Kwon > Use iloc for positional slicing instead of direct slicing in

[jira] [Resolved] (SPARK-32098) Use iloc for positional slicing instead of direct slicing in createDataFrame with Arrow

2020-06-25 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-32098. -- Fix Version/s: 3.1.0 2.4.7 3.0.1 Resolution:

[jira] [Commented] (SPARK-30037) Customize krb5.conf to test Kafka delegation token with MiniKDC

2020-06-25 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17145013#comment-17145013 ] Gabor Somogyi commented on SPARK-30037: ---

[jira] [Commented] (SPARK-30037) Customize krb5.conf to test Kafka delegation token with MiniKDC

2020-06-25 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17145011#comment-17145011 ] Gabor Somogyi commented on SPARK-30037: --- This is done, right? Can we close this? > Customize

[jira] [Commented] (SPARK-32098) Use iloc for positional slicing instead of direct slicing in createDataFrame with Arrow

2020-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17144852#comment-17144852 ] Apache Spark commented on SPARK-32098: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-32098) Use iloc for positional slicing instead of direct slicing in createDataFrame with Arrow

2020-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32098: Assignee: Apache Spark > Use iloc for positional slicing instead of direct slicing in

[jira] [Assigned] (SPARK-32098) Use iloc for positional slicing instead of direct slicing in createDataFrame with Arrow

2020-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32098: Assignee: (was: Apache Spark) > Use iloc for positional slicing instead of direct

[jira] [Commented] (SPARK-32098) Use iloc for positional slicing instead of direct slicing in createDataFrame with Arrow

2020-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17144850#comment-17144850 ] Apache Spark commented on SPARK-32098: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-32099) Remove broken link in cloud integration documentation

2020-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17144834#comment-17144834 ] Apache Spark commented on SPARK-32099: -- User 'guykhazma' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32099) Remove broken link in cloud integration documentation

2020-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32099: Assignee: Apache Spark > Remove broken link in cloud integration documentation >

[jira] [Assigned] (SPARK-32099) Remove broken link in cloud integration documentation

2020-06-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32099: Assignee: (was: Apache Spark) > Remove broken link in cloud integration

[jira] [Updated] (SPARK-32098) Use iloc for positional slicing instead of direct slicing in createDataFrame with Arrow

2020-06-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32098: - Priority: Critical (was: Major) > Use iloc for positional slicing instead of direct slicing in

[jira] [Updated] (SPARK-32098) Use iloc for positional slicing instead of direct slicing in createDataFrame with Arrow

2020-06-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32098: - Labels: correctness (was: ) > Use iloc for positional slicing instead of direct slicing in

[jira] [Updated] (SPARK-32098) Use iloc for positional slicing instead of direct slicing in createDataFrame with Arrow

2020-06-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32098: - Description: When you use floats are index of pandas, it produces a wrong results: {code} >>>

[jira] [Updated] (SPARK-32099) Remove broken link in cloud integration documentation

2020-06-25 Thread Guy Khazma (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guy Khazma updated SPARK-32099: --- Description: The 3rd link in `IBM Cloud Object Storage connector for Apache Spark` is broken. This

[jira] [Created] (SPARK-32099) Remove broken link in cloud integration documentation

2020-06-25 Thread Guy Khazma (Jira)
Guy Khazma created SPARK-32099: -- Summary: Remove broken link in cloud integration documentation Key: SPARK-32099 URL: https://issues.apache.org/jira/browse/SPARK-32099 Project: Spark Issue

[jira] [Created] (SPARK-32098) Use iloc for positional slicing instead of direct slicing in createDataFrame with Arrow

2020-06-25 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-32098: Summary: Use iloc for positional slicing instead of direct slicing in createDataFrame with Arrow Key: SPARK-32098 URL: https://issues.apache.org/jira/browse/SPARK-32098

[jira] [Commented] (SPARK-28489) KafkaOffsetRangeCalculator.getRanges may drop offsets

2020-06-25 Thread Klaus Telenius (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17144819#comment-17144819 ] Klaus Telenius commented on SPARK-28489: Does this affect non-structured streaming? >