[jira] [Updated] (SPARK-35304) [k8s] Though finishing a job, the driver pod is running infinitely

[ https://issues.apache.org/jira/browse/SPARK-35304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keunhyun Oh updated SPARK-35304: Description: Though finishing a job, the driver pod is running infinitely. Executors are terminat

[jira] [Updated] (SPARK-35304) [k8s] Though finishing a job, the driver pod is running infinitely

[ https://issues.apache.org/jira/browse/SPARK-35304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keunhyun Oh updated SPARK-35304: Description: Though finishing a job, the driver pod is running infinitely. Executors are terminat

[jira] [Updated] (SPARK-35304) [k8s] Though finishing a job, the driver pod is running infinitely

[ https://issues.apache.org/jira/browse/SPARK-35304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keunhyun Oh updated SPARK-35304: Issue Type: Bug (was: Question) > [k8s] Though finishing a job, the driver pod is running infinit

[jira] [Created] (SPARK-35304) [k8s] Though finishing a job, the driver pod is running infinitely

Keunhyun Oh created SPARK-35304: --- Summary: [k8s] Though finishing a job, the driver pod is running infinitely Key: SPARK-35304 URL: https://issues.apache.org/jira/browse/SPARK-35304 Project: Spark

[jira] [Assigned] (SPARK-35133) EXPLAIN CODEGEN does not work with AQE

[ https://issues.apache.org/jira/browse/SPARK-35133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35133: Assignee: (was: Apache Spark) > EXPLAIN CODEGEN does not work with AQE >

[jira] [Assigned] (SPARK-35133) EXPLAIN CODEGEN does not work with AQE

[ https://issues.apache.org/jira/browse/SPARK-35133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35133: Assignee: Apache Spark > EXPLAIN CODEGEN does not work with AQE > ---

[jira] [Commented] (SPARK-35133) EXPLAIN CODEGEN does not work with AQE

[ https://issues.apache.org/jira/browse/SPARK-35133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338768#comment-17338768 ] Apache Spark commented on SPARK-35133: -- User 'c21' has created a pull request for t

[jira] [Commented] (SPARK-35303) Enable pinned thread mode by default

[ https://issues.apache.org/jira/browse/SPARK-35303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338738#comment-17338738 ] Apache Spark commented on SPARK-35303: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-35303) Enable pinned thread mode by default

[ https://issues.apache.org/jira/browse/SPARK-35303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35303: Assignee: Apache Spark > Enable pinned thread mode by default > -

[jira] [Commented] (SPARK-35303) Enable pinned thread mode by default

[ https://issues.apache.org/jira/browse/SPARK-35303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338737#comment-17338737 ] Apache Spark commented on SPARK-35303: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-35303) Enable pinned thread mode by default

[ https://issues.apache.org/jira/browse/SPARK-35303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35303: Assignee: (was: Apache Spark) > Enable pinned thread mode by default > --

[jira] [Commented] (SPARK-35302) Benchmark workflow should create new files for new benchmarks

[ https://issues.apache.org/jira/browse/SPARK-35302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338729#comment-17338729 ] Apache Spark commented on SPARK-35302: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-35302) Benchmark workflow should create new files for new benchmarks

[ https://issues.apache.org/jira/browse/SPARK-35302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35302: Assignee: (was: Apache Spark) > Benchmark workflow should create new files for new be

[jira] [Commented] (SPARK-35302) Benchmark workflow should create new files for new benchmarks

[ https://issues.apache.org/jira/browse/SPARK-35302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338730#comment-17338730 ] Apache Spark commented on SPARK-35302: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-35302) Benchmark workflow should create new files for new benchmarks

[ https://issues.apache.org/jira/browse/SPARK-35302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35302: Assignee: Apache Spark > Benchmark workflow should create new files for new benchmarks >

[jira] [Created] (SPARK-35303) Enable pinned thread mode by default

Hyukjin Kwon created SPARK-35303: Summary: Enable pinned thread mode by default Key: SPARK-35303 URL: https://issues.apache.org/jira/browse/SPARK-35303 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-35302) Benchmark workflow should create new files for new benchmarks

Hyukjin Kwon created SPARK-35302: Summary: Benchmark workflow should create new files for new benchmarks Key: SPARK-35302 URL: https://issues.apache.org/jira/browse/SPARK-35302 Project: Spark

[jira] [Updated] (SPARK-35297) Modify the comment about the executor

[ https://issues.apache.org/jira/browse/SPARK-35297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] roryqi updated SPARK-35297: --- Description: Now Spark Executor already can be used in Kubernetes. So we should modify the comment in the Ex

[jira] [Created] (SPARK-35301) Document migration from Koalas to pandas APIs on Spark

Hyukjin Kwon created SPARK-35301: Summary: Document migration from Koalas to pandas APIs on Spark Key: SPARK-35301 URL: https://issues.apache.org/jira/browse/SPARK-35301 Project: Spark Issue

[jira] [Resolved] (SPARK-35300) Standardize module name in install.rst

[ https://issues.apache.org/jira/browse/SPARK-35300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35300. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32427 [https://gi

[jira] [Assigned] (SPARK-35300) Standardize module name in install.rst

[ https://issues.apache.org/jira/browse/SPARK-35300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35300: Assignee: Xinrong Meng > Standardize module name in install.rst > ---

[jira] [Commented] (SPARK-35300) Standardize module name in install.rst

[ https://issues.apache.org/jira/browse/SPARK-35300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338711#comment-17338711 ] Apache Spark commented on SPARK-35300: -- User 'xinrong-databricks' has created a pul

[jira] [Commented] (SPARK-35300) Standardize module name in install.rst

[ https://issues.apache.org/jira/browse/SPARK-35300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338710#comment-17338710 ] Apache Spark commented on SPARK-35300: -- User 'xinrong-databricks' has created a pul

[jira] [Assigned] (SPARK-35300) Standardize module name in install.rst

[ https://issues.apache.org/jira/browse/SPARK-35300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35300: Assignee: (was: Apache Spark) > Standardize module name in install.rst >

[jira] [Assigned] (SPARK-35300) Standardize module name in install.rst

[ https://issues.apache.org/jira/browse/SPARK-35300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35300: Assignee: Apache Spark > Standardize module name in install.rst > ---

[jira] [Created] (SPARK-35300) Standardize module name in install.rst

Xinrong Meng created SPARK-35300: Summary: Standardize module name in install.rst Key: SPARK-35300 URL: https://issues.apache.org/jira/browse/SPARK-35300 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-35297) Modify the comment about the executor

[ https://issues.apache.org/jira/browse/SPARK-35297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] roryqi updated SPARK-35297: --- Summary: Modify the comment about the executor (was: Modify the annotation about the executor) > Modify th

[jira] [Updated] (SPARK-35297) Modify the comment about the executor

[ https://issues.apache.org/jira/browse/SPARK-35297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] roryqi updated SPARK-35297: --- Description: Now Spark Executor already can be used in Kubernetes scheduler. So we should modify the comment

[jira] [Updated] (SPARK-35299) Dataframe overwrite on S3 does not delete old files with S3 object-put to table path

[ https://issues.apache.org/jira/browse/SPARK-35299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yusheng Ding updated SPARK-35299: - Description: To reproduce: test_table path: s3a://test_bucket/test_table/   df = spark_sessio

[jira] [Updated] (SPARK-35299) Dataframe overwrite on S3 does not delete old files with S3 object-put to table path

[ https://issues.apache.org/jira/browse/SPARK-35299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yusheng Ding updated SPARK-35299: - Description: To reproduce: test_table path: s3a://test_bucket/test_table/   df = spark_sessio

[jira] [Updated] (SPARK-35299) Dataframe overwrite on S3 does not delete old files with S3 object-put to table path

[ https://issues.apache.org/jira/browse/SPARK-35299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yusheng Ding updated SPARK-35299: - Description: To reproduce: test_table path: s3a://test_bucket/test_table/   df = spark_sessio

[jira] [Updated] (SPARK-35299) Dataframe overwrite on S3 does not delete old files with S3 object-put to table path

[ https://issues.apache.org/jira/browse/SPARK-35299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yusheng Ding updated SPARK-35299: - Description: To reproduce: test_table path: s3a://test_bucket/test_table/   df = spark_sessio

[jira] [Created] (SPARK-35299) Dataframe overwrite on S3 does not delete old files with S3 object-put to table path

Yusheng Ding created SPARK-35299: Summary: Dataframe overwrite on S3 does not delete old files with S3 object-put to table path Key: SPARK-35299 URL: https://issues.apache.org/jira/browse/SPARK-35299

[jira] [Commented] (SPARK-35207) hash() and other hash builtins do not normalize negative zero

[ https://issues.apache.org/jira/browse/SPARK-35207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338698#comment-17338698 ] Pablo Langa Blanco commented on SPARK-35207: Hi [~tarmstrong] , I have read

[jira] [Assigned] (SPARK-34887) Port/integrate Koalas dependencies into PySpark

[ https://issues.apache.org/jira/browse/SPARK-34887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-34887: Assignee: Xinrong Meng > Port/integrate Koalas dependencies into PySpark > --

[jira] [Resolved] (SPARK-34887) Port/integrate Koalas dependencies into PySpark

[ https://issues.apache.org/jira/browse/SPARK-34887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34887. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32386 [https://gi

[jira] [Resolved] (SPARK-35292) Delete redundant parameter in mypy.ini

[ https://issues.apache.org/jira/browse/SPARK-35292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35292. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32418 [https://gi

[jira] [Assigned] (SPARK-35292) Delete redundant parameter in mypy.ini

[ https://issues.apache.org/jira/browse/SPARK-35292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35292: Assignee: Walid Gara > Delete redundant parameter in mypy.ini >

[jira] [Resolved] (SPARK-35250) SQL DataFrameReader unescapedQuoteHandling parameter is misdocumented

[ https://issues.apache.org/jira/browse/SPARK-35250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35250. -- Fix Version/s: 3.1.2 3.2.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-35250) SQL DataFrameReader unescapedQuoteHandling parameter is misdocumented

[ https://issues.apache.org/jira/browse/SPARK-35250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35250: Assignee: Hyukjin Kwon > SQL DataFrameReader unescapedQuoteHandling parameter is misdocum

[jira] [Updated] (SPARK-35297) Modify the annotation about the executor

[ https://issues.apache.org/jira/browse/SPARK-35297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] roryqi updated SPARK-35297: --- Component/s: (was: Spark Core) Documentation > Modify the annotation about the executor

[jira] [Updated] (SPARK-35298) Migrate to transformWithPruning for rules in optimizer/Optimizer.scala

[ https://issues.apache.org/jira/browse/SPARK-35298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingyi Bu updated SPARK-35298: -- Description: PushXxx rules are handled in SPARK-35077. > Migrate to transformWithPruning for rules in

[jira] [Created] (SPARK-35298) Migrate to transformWithPruning for rules in optimizer/Optimizer.scala

Yingyi Bu created SPARK-35298: - Summary: Migrate to transformWithPruning for rules in optimizer/Optimizer.scala Key: SPARK-35298 URL: https://issues.apache.org/jira/browse/SPARK-35298 Project: Spark

[jira] [Commented] (SPARK-35290) unionByName with null filling fails for some nested structs

[ https://issues.apache.org/jira/browse/SPARK-35290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338565#comment-17338565 ] L. C. Hsieh commented on SPARK-35290: - Thanks [~Kimahriman]. > unionByName with nu

[jira] [Updated] (SPARK-35155) Add rule id to all Analyzer rules in fixed point batches

[ https://issues.apache.org/jira/browse/SPARK-35155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingyi Bu updated SPARK-35155: -- Description: All Analyzer rules that are run in a fixed point batch can be beneficial for the rule-id-

[jira] [Assigned] (SPARK-35297) Modify the annotation about the executor

[ https://issues.apache.org/jira/browse/SPARK-35297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35297: Assignee: Apache Spark > Modify the annotation about the executor > -

[jira] [Assigned] (SPARK-35297) Modify the annotation about the executor

[ https://issues.apache.org/jira/browse/SPARK-35297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35297: Assignee: (was: Apache Spark) > Modify the annotation about the executor > --

[jira] [Commented] (SPARK-35297) Modify the annotation about the executor

[ https://issues.apache.org/jira/browse/SPARK-35297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338560#comment-17338560 ] Apache Spark commented on SPARK-35297: -- User 'jerqi' has created a pull request for

[jira] [Commented] (SPARK-34791) SparkR throws node stack overflow

[ https://issues.apache.org/jira/browse/SPARK-34791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338549#comment-17338549 ] obfuscated_dvlper commented on SPARK-34791: --- sorry about reporting this late.

[jira] [Created] (SPARK-35297) Modify the annotation about the executor

roryqi created SPARK-35297: -- Summary: Modify the annotation about the executor Key: SPARK-35297 URL: https://issues.apache.org/jira/browse/SPARK-35297 Project: Spark Issue Type: Documentation

[jira] [Commented] (SPARK-35290) unionByName with null filling fails for some nested structs

[ https://issues.apache.org/jira/browse/SPARK-35290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338526#comment-17338526 ] Adam Binford commented on SPARK-35290: -- I've also been playing around with rewritin

[jira] [Commented] (SPARK-35155) Add rule id to all ResolveXxx rules

[ https://issues.apache.org/jira/browse/SPARK-35155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338481#comment-17338481 ] Apache Spark commented on SPARK-35155: -- User 'sigmod' has created a pull request fo

[jira] [Assigned] (SPARK-35155) Add rule id to all ResolveXxx rules

[ https://issues.apache.org/jira/browse/SPARK-35155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35155: Assignee: (was: Apache Spark) > Add rule id to all ResolveXxx rules > ---

[jira] [Assigned] (SPARK-35155) Add rule id to all ResolveXxx rules

[ https://issues.apache.org/jira/browse/SPARK-35155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35155: Assignee: Apache Spark > Add rule id to all ResolveXxx rules > --

[jira] [Commented] (SPARK-35155) Add rule id to all ResolveXxx rules

[ https://issues.apache.org/jira/browse/SPARK-35155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338480#comment-17338480 ] Apache Spark commented on SPARK-35155: -- User 'sigmod' has created a pull request fo

[jira] [Commented] (SPARK-35290) unionByName with null filling fails for some nested structs

[ https://issues.apache.org/jira/browse/SPARK-35290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338477#comment-17338477 ] L. C. Hsieh commented on SPARK-35290: - I will take a look. > unionByName with null

[jira] [Commented] (SPARK-35290) unionByName with null filling fails for some nested structs

[ https://issues.apache.org/jira/browse/SPARK-35290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338476#comment-17338476 ] L. C. Hsieh commented on SPARK-35290: - Thanks [~hyukjin.kwon] for ping me. > unionB

[jira] [Commented] (SPARK-35296) Dataset.observe fails with an assertion

[ https://issues.apache.org/jira/browse/SPARK-35296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338465#comment-17338465 ] Tanel Kiis commented on SPARK-35296: I finally managed to change the UT in such way,

[jira] [Updated] (SPARK-35296) Dataset.observe fails with an assertion

[ https://issues.apache.org/jira/browse/SPARK-35296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanel Kiis updated SPARK-35296: --- Attachment: 2021-05-03_18-34.png > Dataset.observe fails with an assertion > ---

[jira] [Updated] (SPARK-35296) Dataset.observe fails with an assertion

[ https://issues.apache.org/jira/browse/SPARK-35296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanel Kiis updated SPARK-35296: --- Description: I hit this assertion error when using dataset.observe: {code} java.lang.AssertionError:

[jira] [Updated] (SPARK-35296) Dataset.observe fails with an assertion

[ https://issues.apache.org/jira/browse/SPARK-35296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanel Kiis updated SPARK-35296: --- Description: I hit this assertion error when using dataset.observe: {code} java.lang.AssertionError:

[jira] [Comment Edited] (SPARK-35296) Dataset.observe fails with an assertion

[ https://issues.apache.org/jira/browse/SPARK-35296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338438#comment-17338438 ] Tanel Kiis edited comment on SPARK-35296 at 5/3/21, 3:58 PM: -

[jira] [Commented] (SPARK-35296) Dataset.observe fails with an assertion

[ https://issues.apache.org/jira/browse/SPARK-35296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338441#comment-17338441 ] Tanel Kiis commented on SPARK-35296: [~hvanhovell] The assertion in AggregatingAccum

[jira] [Commented] (SPARK-35296) Dataset.observe fails with an assertion

[ https://issues.apache.org/jira/browse/SPARK-35296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338438#comment-17338438 ] Tanel Kiis commented on SPARK-35296: I tried to change an excisting UT to reproduce

[jira] [Updated] (SPARK-35296) Dataset.observe fails with an assertion

[ https://issues.apache.org/jira/browse/SPARK-35296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanel Kiis updated SPARK-35296: --- Description: I hit this assertion error when using dataset.observe: {code} java.lang.AssertionError:

[jira] [Updated] (SPARK-35296) Dataset.observe fails with an assertion

[ https://issues.apache.org/jira/browse/SPARK-35296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanel Kiis updated SPARK-35296: --- Description: I hit this assertion error when using dataset.observe: {code} java.lang.AssertionError:

[jira] [Updated] (SPARK-35296) Dataset.observe fails with an assertion

[ https://issues.apache.org/jira/browse/SPARK-35296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanel Kiis updated SPARK-35296: --- Description: I hit this assertion error when using dataset.observe: {code} {code} was: I hit this

[jira] [Updated] (SPARK-35296) Dataset.observe fails with an assertion

[ https://issues.apache.org/jira/browse/SPARK-35296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanel Kiis updated SPARK-35296: --- Description: I hit this assertion error when using dataset.observe: {code} java.lang.AssertionError:

[jira] [Created] (SPARK-35296) Dataset.observe fails with an assertion

Tanel Kiis created SPARK-35296: -- Summary: Dataset.observe fails with an assertion Key: SPARK-35296 URL: https://issues.apache.org/jira/browse/SPARK-35296 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-34794) Nested higher-order functions broken in DSL

[ https://issues.apache.org/jira/browse/SPARK-34794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338411#comment-17338411 ] Apache Spark commented on SPARK-34794: -- User 'maropu' has created a pull request fo

[jira] [Assigned] (SPARK-35295) Replace fully com.github.fommil.netlib by dev.ludovic.netlib:2.0

[ https://issues.apache.org/jira/browse/SPARK-35295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35295: Assignee: Apache Spark > Replace fully com.github.fommil.netlib by dev.ludovic.netlib:2.0

[jira] [Assigned] (SPARK-35295) Replace fully com.github.fommil.netlib by dev.ludovic.netlib:2.0

[ https://issues.apache.org/jira/browse/SPARK-35295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35295: Assignee: (was: Apache Spark) > Replace fully com.github.fommil.netlib by dev.ludovic

[jira] [Commented] (SPARK-35295) Replace fully com.github.fommil.netlib by dev.ludovic.netlib:2.0

[ https://issues.apache.org/jira/browse/SPARK-35295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338334#comment-17338334 ] Apache Spark commented on SPARK-35295: -- User 'luhenry' has created a pull request f

[jira] [Created] (SPARK-35295) Replace fully com.github.fommil.netlib by dev.ludovic.netlib:2.0

Ludovic Henry created SPARK-35295: - Summary: Replace fully com.github.fommil.netlib by dev.ludovic.netlib:2.0 Key: SPARK-35295 URL: https://issues.apache.org/jira/browse/SPARK-35295 Project: Spark

[jira] [Assigned] (SPARK-35250) SQL DataFrameReader unescapedQuoteHandling parameter is misdocumented

[ https://issues.apache.org/jira/browse/SPARK-35250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35250: Assignee: (was: Apache Spark) > SQL DataFrameReader unescapedQuoteHandling parameter

[jira] [Commented] (SPARK-35250) SQL DataFrameReader unescapedQuoteHandling parameter is misdocumented

[ https://issues.apache.org/jira/browse/SPARK-35250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338323#comment-17338323 ] Apache Spark commented on SPARK-35250: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-35250) SQL DataFrameReader unescapedQuoteHandling parameter is misdocumented

[ https://issues.apache.org/jira/browse/SPARK-35250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35250: Assignee: Apache Spark > SQL DataFrameReader unescapedQuoteHandling parameter is misdocum

[jira] [Commented] (SPARK-35250) SQL DataFrameReader unescapedQuoteHandling parameter is misdocumented

[ https://issues.apache.org/jira/browse/SPARK-35250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338319#comment-17338319 ] Hyukjin Kwon commented on SPARK-35250: -- NVM, let me just make a quick fix :-) > SQ

[jira] [Commented] (SPARK-35108) Pickle produces incorrect key labels for GenericRowWithSchema (data corruption)

[ https://issues.apache.org/jira/browse/SPARK-35108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338312#comment-17338312 ] Hyukjin Kwon commented on SPARK-35108: -- [~revans2] and [~tgraves] can you confirm t

[jira] [Commented] (SPARK-35108) Pickle produces incorrect key labels for GenericRowWithSchema (data corruption)

[ https://issues.apache.org/jira/browse/SPARK-35108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338311#comment-17338311 ] Hyukjin Kwon commented on SPARK-35108: -- I think this is a duplicate of SPARK-34545.

[jira] [Commented] (SPARK-35272) org.apache.spark.SparkException: Task not serializable

[ https://issues.apache.org/jira/browse/SPARK-35272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338308#comment-17338308 ] Hyukjin Kwon commented on SPARK-35272: -- Ohh, I just noticed that {{obj}} is not inc

[jira] [Resolved] (SPARK-35237) In k8s, during running spark job, IllegalArgumentException(too large frame) is raised on spark driver. It seems to related to prometheus

[ https://issues.apache.org/jira/browse/SPARK-35237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35237. -- Resolution: Invalid It's best to ask questions into mailing lists. Let's loop them before we f

[jira] [Resolved] (SPARK-35249) to_timestamp can't parse 6 digit microsecond SSSSSS

[ https://issues.apache.org/jira/browse/SPARK-35249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35249. -- Resolution: Cannot Reproduce > to_timestamp can't parse 6 digit microsecond SS > -

[jira] [Commented] (SPARK-35250) SQL DataFrameReader unescapedQuoteHandling parameter is misdocumented

[ https://issues.apache.org/jira/browse/SPARK-35250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338304#comment-17338304 ] Hyukjin Kwon commented on SPARK-35250: -- Are you interested in submitting a PR, [~ti

[jira] [Commented] (SPARK-35250) SQL DataFrameReader unescapedQuoteHandling parameter is misdocumented

[ https://issues.apache.org/jira/browse/SPARK-35250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338305#comment-17338305 ] Hyukjin Kwon commented on SPARK-35250: -- cc [~LuciferYang] FYI > SQL DataFrameReade

[jira] [Commented] (SPARK-35251) Improve LiveEntityHelpers.newAccumulatorInfos performace

[ https://issues.apache.org/jira/browse/SPARK-35251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338303#comment-17338303 ] Hyukjin Kwon commented on SPARK-35251: -- Can you show reproducer and how you managed

[jira] [Updated] (SPARK-35250) SQL DataFrameReader unescapedQuoteHandling parameter is misdocumented

[ https://issues.apache.org/jira/browse/SPARK-35250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-35250: - Labels: starter (was: GoodForNewContributors easy-fix) > SQL DataFrameReader unescapedQuoteHand

[jira] [Commented] (SPARK-35252) PartitionReaderFactory's Implemention Class of DataSourceV2: sqlConf parameter is null

[ https://issues.apache.org/jira/browse/SPARK-35252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338302#comment-17338302 ] Hyukjin Kwon commented on SPARK-35252: -- You can use {{SQLConf.get}} instead. > Par

[jira] [Comment Edited] (SPARK-35272) org.apache.spark.SparkException: Task not serializable

[ https://issues.apache.org/jira/browse/SPARK-35272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338301#comment-17338301 ] Sandeep Katta edited comment on SPARK-35272 at 5/3/21, 10:41 AM: -

[jira] [Commented] (SPARK-35272) org.apache.spark.SparkException: Task not serializable

[ https://issues.apache.org/jira/browse/SPARK-35272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338301#comment-17338301 ] Sandeep Katta commented on SPARK-35272: --- That is correct if I am broadcasting *Non

[jira] [Commented] (SPARK-35272) org.apache.spark.SparkException: Task not serializable

[ https://issues.apache.org/jira/browse/SPARK-35272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338299#comment-17338299 ] Hyukjin Kwon commented on SPARK-35272: -- It should be serializable. Spark cannot mak

[jira] [Resolved] (SPARK-35265) abs return negative

[ https://issues.apache.org/jira/browse/SPARK-35265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35265. -- Resolution: Invalid > abs return negative > --- > > Key: SPARK

[jira] [Commented] (SPARK-35265) abs return negative

[ https://issues.apache.org/jira/browse/SPARK-35265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338298#comment-17338298 ] Hyukjin Kwon commented on SPARK-35265: -- This is fixed via {{spark.sql.ansi.enabled}

[jira] [Updated] (SPARK-35262) Memory leak when dataset is being persisted

[ https://issues.apache.org/jira/browse/SPARK-35262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-35262: - Priority: Major (was: Critical) > Memory leak when dataset is being persisted > ---

[jira] [Comment Edited] (SPARK-35272) org.apache.spark.SparkException: Task not serializable

[ https://issues.apache.org/jira/browse/SPARK-35272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338297#comment-17338297 ] Sandeep Katta edited comment on SPARK-35272 at 5/3/21, 10:30 AM: -

[jira] [Commented] (SPARK-35272) org.apache.spark.SparkException: Task not serializable

[ https://issues.apache.org/jira/browse/SPARK-35272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338297#comment-17338297 ] Sandeep Katta commented on SPARK-35272: --- This is the minimal code with which we ca

[jira] [Commented] (SPARK-35267) nullable field is set to false for integer type when using reflection to get StructType for a case class

[ https://issues.apache.org/jira/browse/SPARK-35267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338296#comment-17338296 ] Hyukjin Kwon commented on SPARK-35267: -- It's because {{Int}} cannot be {{null}}, ri

[jira] [Resolved] (SPARK-35267) nullable field is set to false for integer type when using reflection to get StructType for a case class

[ https://issues.apache.org/jira/browse/SPARK-35267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35267. -- Resolution: Invalid > nullable field is set to false for integer type when using reflection to

[jira] [Commented] (SPARK-33534) Allow specifying a minimum number of bytes in a split of a file

[ https://issues.apache.org/jira/browse/SPARK-33534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338295#comment-17338295 ] Niels Basjes commented on SPARK-33534: -- [~Suhass]  To be clear: I wrote this tool t

[jira] [Commented] (SPARK-35272) org.apache.spark.SparkException: Task not serializable

[ https://issues.apache.org/jira/browse/SPARK-35272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338293#comment-17338293 ] Hyukjin Kwon commented on SPARK-35272: -- Why don't you make {{NonSerializable}} exte

  1   2   >