[jira] [Commented] (SPARK-48180) Analyzer bug with multiple ORDER BY items for input table argument

2024-05-07 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844471#comment-17844471 ] Daniel commented on SPARK-48180: Fix here: [https://github.com/apache/spark/pull/46451] > Analyzer bug

[jira] [Commented] (SPARK-48180) Analyzer bug with multiple ORDER BY items for input table argument

2024-05-07 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844467#comment-17844467 ] Daniel commented on SPARK-48180: The bug is that parentheses are required around the two arguments in

[jira] [Created] (SPARK-48180) Analyzer bug with multiple ORDER BY items for input table argument

2024-05-07 Thread Daniel (Jira)
Daniel created SPARK-48180: -- Summary: Analyzer bug with multiple ORDER BY items for input table argument Key: SPARK-48180 URL: https://issues.apache.org/jira/browse/SPARK-48180 Project: Spark

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-30 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/30/24 7:11 PM: - Edit: I updated this

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-30 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/30/24 6:57 PM: - Edit: I updated this

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-30 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/30/24 6:43 PM: - Edit: I updated this

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-30 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/30/24 5:55 PM: - Edit: I updated this

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 11:11 PM: -- Edit: I updated this

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 10:21 PM: -- Edit: I updated this

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 10:19 PM: -- Edit: I updated this

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 10:01 PM: -- Edit: I updated this

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 9:58 PM: - Edit: I updated this

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 9:44 PM: - Edit: I updated this

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 9:38 PM: - Edit: I updated this

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 9:26 PM: - Edit: I updated this

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 9:15 PM: - Edit: I updated this

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 8:54 PM: - Edit: I updated this

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 8:23 PM: - Edit: I updated this

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 8:07 PM: - Edit: I updated this

[jira] [Comment Edited] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835552#comment-17835552 ] Daniel edited comment on SPARK-47578 at 4/29/24 4:50 PM: - Edit: I updated this

[jira] [Commented] (SPARK-47585) SQL core: Migrate logInfo with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842094#comment-17842094 ] Daniel commented on SPARK-47585: [~panbingkun] here are the affected files: spark$ grep logWarning

[jira] [Commented] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842092#comment-17842092 ] Daniel commented on SPARK-47578: Sounds good, I start on this one now > Spark core: Migrate logWarn

[jira] [Comment Edited] (SPARK-47597) Streaming: Migrate logInfo with variables to structured logging framework

2024-04-23 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17840213#comment-17840213 ] Daniel edited comment on SPARK-47597 at 4/23/24 9:44 PM: - spark$ grep logInfo

[jira] [Comment Edited] (SPARK-47597) Streaming: Migrate logInfo with variables to structured logging framework

2024-04-23 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17840213#comment-17840213 ] Daniel edited comment on SPARK-47597 at 4/23/24 9:14 PM: - spark$ grep logInfo

[jira] [Comment Edited] (SPARK-47597) Streaming: Migrate logInfo with variables to structured logging framework

2024-04-23 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17840213#comment-17840213 ] Daniel edited comment on SPARK-47597 at 4/23/24 9:04 PM: - spark$ grep logInfo

[jira] [Comment Edited] (SPARK-47597) Streaming: Migrate logInfo with variables to structured logging framework

2024-04-23 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17840213#comment-17840213 ] Daniel edited comment on SPARK-47597 at 4/23/24 8:31 PM: - spark$ grep logInfo

[jira] [Comment Edited] (SPARK-47597) Streaming: Migrate logInfo with variables to structured logging framework

2024-04-23 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17840213#comment-17840213 ] Daniel edited comment on SPARK-47597 at 4/23/24 8:27 PM: - spark$ grep logInfo

[jira] [Commented] (SPARK-47597) Streaming: Migrate logInfo with variables to structured logging framework

2024-04-23 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17840213#comment-17840213 ] Daniel commented on SPARK-47597: spark$ grep logInfo sql/core/src/main/* -R | cut -d ':' -f 1 | sort -u

[jira] [Commented] (SPARK-47578) Spark core: Migrate logWarn with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835552#comment-17835552 ] Daniel commented on SPARK-47578: spark-2$ grep logWarning sql/core/src/main/* -R | cut -d ':' -f 1 |

[jira] [Comment Edited] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835546#comment-17835546 ] Daniel edited comment on SPARK-47583 at 4/9/24 11:26 PM: - spark$ grep logError

[jira] [Comment Edited] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835546#comment-17835546 ] Daniel edited comment on SPARK-47583 at 4/9/24 11:16 PM: - spark$ grep logError

[jira] [Comment Edited] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835546#comment-17835546 ] Daniel edited comment on SPARK-47583 at 4/9/24 11:14 PM: - spark$ grep logError

[jira] [Comment Edited] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835546#comment-17835546 ] Daniel edited comment on SPARK-47583 at 4/9/24 11:14 PM: - spark$ grep logError

[jira] [Comment Edited] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835546#comment-17835546 ] Daniel edited comment on SPARK-47583 at 4/9/24 11:07 PM: - spark$ grep logError

[jira] [Comment Edited] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835546#comment-17835546 ] Daniel edited comment on SPARK-47583 at 4/9/24 11:07 PM: - spark$ grep logError

[jira] [Comment Edited] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835546#comment-17835546 ] Daniel edited comment on SPARK-47583 at 4/9/24 10:59 PM: - spark$ grep logError

[jira] [Comment Edited] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835546#comment-17835546 ] Daniel edited comment on SPARK-47583 at 4/9/24 10:55 PM: - spark$ grep logError

[jira] [Comment Edited] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835546#comment-17835546 ] Daniel edited comment on SPARK-47583 at 4/9/24 10:53 PM: - spark$ grep logError

[jira] [Comment Edited] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835546#comment-17835546 ] Daniel edited comment on SPARK-47583 at 4/9/24 10:47 PM: - spark$ grep logError

[jira] [Comment Edited] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835546#comment-17835546 ] Daniel edited comment on SPARK-47583 at 4/9/24 10:32 PM: - spark$ grep logError

[jira] [Comment Edited] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835546#comment-17835546 ] Daniel edited comment on SPARK-47583 at 4/9/24 10:30 PM: - spark$ grep logError

[jira] [Commented] (SPARK-47583) SQL core: Migrate logError with variables to structured logging framework

2024-04-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835546#comment-17835546 ] Daniel commented on SPARK-47583: spark$ grep logError sql/core/src/main/* -R | cut -d ':' -f 1 | sort -u

[jira] [Updated] (SPARK-47509) Incorrect results for LambdaFunctions or HigherOrderFunctions in subquery expression plans

2024-03-21 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-47509: --- Summary: Incorrect results for LambdaFunctions or HigherOrderFunctions in subquery expression plans (was:

[jira] [Updated] (SPARK-47509) Incorrect results for LambdaFunction or HigherOrderFunction within subquery expressions

2024-03-21 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-47509: --- Summary: Incorrect results for LambdaFunction or HigherOrderFunction within subquery expressions (was:

[jira] [Created] (SPARK-47509) Incorrect results for LambdaFunction or HigherOrderFunction with subquery expressions

2024-03-21 Thread Daniel (Jira)
Daniel created SPARK-47509: -- Summary: Incorrect results for LambdaFunction or HigherOrderFunction with subquery expressions Key: SPARK-47509 URL: https://issues.apache.org/jira/browse/SPARK-47509 Project:

[jira] [Resolved] (SPARK-47032) Create API for 'analyze' method to send input column(s) to output table unchanged

2024-03-04 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel resolved SPARK-47032. Resolution: Won't Fix After investigating this, we found that any performance benefits from skipping

[jira] [Created] (SPARK-47214) Create API for 'analyze' method to differentiate constant NULL arguments and other types of arguments

2024-02-28 Thread Daniel (Jira)
Daniel created SPARK-47214: -- Summary: Create API for 'analyze' method to differentiate constant NULL arguments and other types of arguments Key: SPARK-47214 URL: https://issues.apache.org/jira/browse/SPARK-47214

[jira] [Closed] (SPARK-46638) Create API to acquire execution memory for 'eval' and 'terminate' methods

2024-02-13 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel closed SPARK-46638. -- > Create API to acquire execution memory for 'eval' and 'terminate' methods >

[jira] [Created] (SPARK-47032) Create API for 'analyze' method to send input column(s) to output table unchanged

2024-02-13 Thread Daniel (Jira)
Daniel created SPARK-47032: -- Summary: Create API for 'analyze' method to send input column(s) to output table unchanged Key: SPARK-47032 URL: https://issues.apache.org/jira/browse/SPARK-47032 Project: Spark

[jira] [Created] (SPARK-47002) Enforce that 'AnalyzeResult' 'orderBy' field is a list of pyspark.sql.functions.OrderingColumn

2024-02-07 Thread Daniel (Jira)
Daniel created SPARK-47002: -- Summary: Enforce that 'AnalyzeResult' 'orderBy' field is a list of pyspark.sql.functions.OrderingColumn Key: SPARK-47002 URL: https://issues.apache.org/jira/browse/SPARK-47002

[jira] [Created] (SPARK-46966) Create API for 'analyze' method to indicate subset of input table columns to select

2024-02-02 Thread Daniel (Jira)
Daniel created SPARK-46966: -- Summary: Create API for 'analyze' method to indicate subset of input table columns to select Key: SPARK-46966 URL: https://issues.apache.org/jira/browse/SPARK-46966 Project:

[jira] [Resolved] (SPARK-46638) Create API to acquire execution memory for 'eval' and 'terminate' methods

2024-02-02 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel resolved SPARK-46638. Resolution: Won't Fix > Create API to acquire execution memory for 'eval' and 'terminate' methods >

[jira] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-01-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890 ] Daniel deleted comment on SPARK-46890: was (Author: JIRAUSER285772): I think this `tokenIndexArr` within Spark's `UnivocityParser` class has different values in the passing and failing cases:

[jira] [Commented] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-01-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17812072#comment-17812072 ] Daniel commented on SPARK-46890: [~maxgekk] I created a bug fix here:

[jira] [Comment Edited] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-01-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17812059#comment-17812059 ] Daniel edited comment on SPARK-46890 at 1/29/24 9:27 PM: - I think this

[jira] [Commented] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-01-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17812059#comment-17812059 ] Daniel commented on SPARK-46890: I think this `tokenIndexArr` within Spark's `UnivocityParser` class has

[jira] [Commented] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-01-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17812058#comment-17812058 ] Daniel commented on SPARK-46890: The bug happens when the Univocity parser is converting the parsed

[jira] [Updated] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-01-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-46890: --- Attachment: image-2024-01-29-13-22-05-326.png > CSV fails on a column with default and without enforcing

[jira] [Commented] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-01-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17812051#comment-17812051 ] Daniel commented on SPARK-46890: The exception comes from here:

[jira] [Commented] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-01-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17812047#comment-17812047 ] Daniel commented on SPARK-46890: Thanks [~maxgekk] both of the above tests reproduce the bug now. I will

[jira] [Commented] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-01-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17812040#comment-17812040 ] Daniel commented on SPARK-46890: I tried the exact command in the bug description, and it doesn't cause

[jira] [Commented] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-01-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17812034#comment-17812034 ] Daniel commented on SPARK-46890: This unit test does not seem to reproduce the problem:   {code:java}

[jira] [Comment Edited] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-01-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17812034#comment-17812034 ] Daniel edited comment on SPARK-46890 at 1/29/24 7:31 PM: - This unit test does

[jira] [Commented] (SPARK-46890) CSV fails on a column with default and without enforcing schema

2024-01-29 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17812002#comment-17812002 ] Daniel commented on SPARK-46890: Thanks [~maxgekk] for writing down the details here. It looks like this

[jira] [Updated] (SPARK-46879) Run optimizer on REPLACE TABLE column defaults

2024-01-26 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-46879: --- Summary: Run optimizer on REPLACE TABLE column defaults (was: Run optimizer on CREATE TABLE column

[jira] [Created] (SPARK-46879) Run optimizer on CREATE TABLE column defaults

2024-01-26 Thread Daniel (Jira)
Daniel created SPARK-46879: -- Summary: Run optimizer on CREATE TABLE column defaults Key: SPARK-46879 URL: https://issues.apache.org/jira/browse/SPARK-46879 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-46849) Run optimizer on CREATE TABLE column defaults

2024-01-24 Thread Daniel (Jira)
Daniel created SPARK-46849: -- Summary: Run optimizer on CREATE TABLE column defaults Key: SPARK-46849 URL: https://issues.apache.org/jira/browse/SPARK-46849 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-46638) Create API to acquire execution memory for 'eval' and 'terminate' methods

2024-01-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-46638: --- Summary: Create API to acquire execution memory for 'eval' and 'terminate' methods (was: Create API to

[jira] [Created] (SPARK-46638) Create API to acquire execution memory for eval and terminate methods

2024-01-09 Thread Daniel (Jira)
Daniel created SPARK-46638: -- Summary: Create API to acquire execution memory for eval and terminate methods Key: SPARK-46638 URL: https://issues.apache.org/jira/browse/SPARK-46638 Project: Spark

[jira] [Commented] (SPARK-46370) Fix bug when querying from table after changing defaults

2023-12-11 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17795514#comment-17795514 ] Daniel commented on SPARK-46370: The bug is here:   {{drop table if exists t;}} {{create table t(i

[jira] [Created] (SPARK-46370) Fix bug when querying from table after changing defaults

2023-12-11 Thread Daniel (Jira)
Daniel created SPARK-46370: -- Summary: Fix bug when querying from table after changing defaults Key: SPARK-46370 URL: https://issues.apache.org/jira/browse/SPARK-46370 Project: Spark Issue Type:

[jira] [Created] (SPARK-46040) Update API for 'analyze' partitioning/ordering columns to support general expressions

2023-11-21 Thread Daniel (Jira)
Daniel created SPARK-46040: -- Summary: Update API for 'analyze' partitioning/ordering columns to support general expressions Key: SPARK-46040 URL: https://issues.apache.org/jira/browse/SPARK-46040 Project:

[jira] [Created] (SPARK-45810) Create API to stop consuming rows from the input table

2023-11-06 Thread Daniel (Jira)
Daniel created SPARK-45810: -- Summary: Create API to stop consuming rows from the input table Key: SPARK-45810 URL: https://issues.apache.org/jira/browse/SPARK-45810 Project: Spark Issue Type:

[jira] [Created] (SPARK-45746) Return specific error messages if UDTF 'analyze' method accepts or returns wrong values

2023-10-31 Thread Daniel (Jira)
Daniel created SPARK-45746: -- Summary: Return specific error messages if UDTF 'analyze' method accepts or returns wrong values Key: SPARK-45746 URL: https://issues.apache.org/jira/browse/SPARK-45746 Project:

[jira] [Created] (SPARK-45523) Return useful error message if UDTF returns None for non-nullable column

2023-10-12 Thread Daniel (Jira)
Daniel created SPARK-45523: -- Summary: Return useful error message if UDTF returns None for non-nullable column Key: SPARK-45523 URL: https://issues.apache.org/jira/browse/SPARK-45523 Project: Spark

[jira] [Created] (SPARK-45402) Add API for 'analyze' method to return a buffer to be consumed on each class creation

2023-10-03 Thread Daniel (Jira)
Daniel created SPARK-45402: -- Summary: Add API for 'analyze' method to return a buffer to be consumed on each class creation Key: SPARK-45402 URL: https://issues.apache.org/jira/browse/SPARK-45402 Project:

[jira] [Created] (SPARK-45362) Project out PARTITION BY expressions before 'eval' method consumes input rows

2023-09-27 Thread Daniel (Jira)
Daniel created SPARK-45362: -- Summary: Project out PARTITION BY expressions before 'eval' method consumes input rows Key: SPARK-45362 URL: https://issues.apache.org/jira/browse/SPARK-45362 Project: Spark

[jira] [Created] (SPARK-44901) Add API in 'analyze' method to return partitioning/ordering expressions

2023-08-21 Thread Daniel (Jira)
Daniel created SPARK-44901: -- Summary: Add API in 'analyze' method to return partitioning/ordering expressions Key: SPARK-44901 URL: https://issues.apache.org/jira/browse/SPARK-44901 Project: Spark

[jira] [Commented] (SPARK-44748) Query execution to support PARTITION BY and ORDER BY clause for table arguments

2023-08-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752531#comment-17752531 ] Daniel commented on SPARK-44748: I can work on this one > Query execution to support PARTITION BY and

[jira] [Resolved] (SPARK-44503) Query planning to support PARTITION BY and ORDER BY clause for table arguments

2023-08-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel resolved SPARK-44503. Resolution: Fixed > Query planning to support PARTITION BY and ORDER BY clause for table arguments >

[jira] [Created] (SPARK-44748) Query execution to support PARTITION BY and ORDER BY clause for table arguments

2023-08-09 Thread Daniel (Jira)
Daniel created SPARK-44748: -- Summary: Query execution to support PARTITION BY and ORDER BY clause for table arguments Key: SPARK-44748 URL: https://issues.apache.org/jira/browse/SPARK-44748 Project: Spark

[jira] [Updated] (SPARK-44503) Query planning to support PARTITION BY and ORDER BY clause for table arguments

2023-08-09 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-44503: --- Summary: Query planning to support PARTITION BY and ORDER BY clause for table arguments (was: Support

[jira] [Commented] (SPARK-44503) Support PARTITION BY and ORDER BY clause for table arguments

2023-08-02 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750505#comment-17750505 ] Daniel commented on SPARK-44503: Here is where we call the "eval" method on each input row that the UDTF

[jira] [Comment Edited] (SPARK-44503) Support PARTITION BY and ORDER BY clause for table arguments

2023-08-02 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750505#comment-17750505 ] Daniel edited comment on SPARK-44503 at 8/2/23 11:51 PM: - Here is where we call

[jira] [Reopened] (SPARK-44503) Support PARTITION BY and ORDER BY clause for table arguments

2023-07-26 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel reopened SPARK-44503: Reopening since I added the SQL grammar support only in [https://github.com/apache/spark/pull/42100,] and

[jira] [Commented] (SPARK-44503) Support PARTITION BY and ORDER BY clause for table arguments

2023-07-20 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745258#comment-17745258 ] Daniel commented on SPARK-44503: I can work on this part > Support PARTITION BY and ORDER BY clause for

[jira] [Created] (SPARK-44503) Support PARTITION BY and ORDER BY clause for table arguments

2023-07-20 Thread Daniel (Jira)
Daniel created SPARK-44503: -- Summary: Support PARTITION BY and ORDER BY clause for table arguments Key: SPARK-44503 URL: https://issues.apache.org/jira/browse/SPARK-44503 Project: Spark Issue

[jira] [Created] (SPARK-44395) Update table function arguments to require parentheses around identifier after the TABLE keyword

2023-07-12 Thread Daniel (Jira)
Daniel created SPARK-44395: -- Summary: Update table function arguments to require parentheses around identifier after the TABLE keyword Key: SPARK-44395 URL: https://issues.apache.org/jira/browse/SPARK-44395

[jira] [Created] (SPARK-44123) Clear persistent objects before SQL analyzer golden file tests

2023-06-20 Thread Daniel (Jira)
Daniel created SPARK-44123: -- Summary: Clear persistent objects before SQL analyzer golden file tests Key: SPARK-44123 URL: https://issues.apache.org/jira/browse/SPARK-44123 Project: Spark Issue

[jira] [Created] (SPARK-43986) Add error classes for HyperLogLog functions

2023-06-06 Thread Daniel (Jira)
Daniel created SPARK-43986: -- Summary: Add error classes for HyperLogLog functions Key: SPARK-43986 URL: https://issues.apache.org/jira/browse/SPARK-43986 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-43529) Support general expressions as OPTIONS values

2023-05-16 Thread Daniel (Jira)
Daniel created SPARK-43529: -- Summary: Support general expressions as OPTIONS values Key: SPARK-43529 URL: https://issues.apache.org/jira/browse/SPARK-43529 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-43358) Cover the resolution of insertion with the SupportsCustomSchemaWrite interface

2023-05-03 Thread Daniel (Jira)
Daniel created SPARK-43358: -- Summary: Cover the resolution of insertion with the SupportsCustomSchemaWrite interface Key: SPARK-43358 URL: https://issues.apache.org/jira/browse/SPARK-43358 Project: Spark

[jira] [Created] (SPARK-43313) Adding missing default values for MERGE INSERT actions

2023-04-27 Thread Daniel (Jira)
Daniel created SPARK-43313: -- Summary: Adding missing default values for MERGE INSERT actions Key: SPARK-43313 URL: https://issues.apache.org/jira/browse/SPARK-43313 Project: Spark Issue Type:

[jira] [Updated] (SPARK-43085) Fix bug in column DEFAULT assignment for target tables with multi-part names

2023-04-10 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-43085: --- Summary: Fix bug in column DEFAULT assignment for target tables with multi-part names (was: Fix bug in

[jira] [Updated] (SPARK-43085) Fix bug in column DEFAULT assignment for target tables with multi-part names

2023-04-10 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-43085: --- Description: (was: To reproduce: {{CREATE DATABASE If NOT EXISTS main.codydemos;}} {{CREATE OR REPLACE

[jira] [Updated] (SPARK-43085) Fix bug in column DEFAULT assignment for target tables with three-part names

2023-04-10 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-43085: --- Description: To reproduce: {{CREATE DATABASE If NOT EXISTS main.codydemos;}} {{CREATE OR REPLACE TABLE

[jira] [Created] (SPARK-43085) Fix bug in column DEFAULT assignment for target tables with three-part names

2023-04-10 Thread Daniel (Jira)
Daniel created SPARK-43085: -- Summary: Fix bug in column DEFAULT assignment for target tables with three-part names Key: SPARK-43085 URL: https://issues.apache.org/jira/browse/SPARK-43085 Project: Spark

[jira] [Updated] (SPARK-43085) Fix bug in column DEFAULT assignment for target tables with three-part names

2023-04-10 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-43085: --- Description: To reproduce:   {{CREATE DATABASE If NOT EXISTS main.codydemos;}} {{CREATE OR REPLACE TABLE

[jira] [Created] (SPARK-43071) Support SELECT DEFAULT with ORDER BY, LIMIT, OFFSET for INSERT source relation

2023-04-07 Thread Daniel (Jira)
Daniel created SPARK-43071: -- Summary: Support SELECT DEFAULT with ORDER BY, LIMIT, OFFSET for INSERT source relation Key: SPARK-43071 URL: https://issues.apache.org/jira/browse/SPARK-43071 Project: Spark

[jira] [Created] (SPARK-43018) Fix bug with timestamp literals

2023-04-03 Thread Daniel (Jira)
Daniel created SPARK-43018: -- Summary: Fix bug with timestamp literals Key: SPARK-43018 URL: https://issues.apache.org/jira/browse/SPARK-43018 Project: Spark Issue Type: Sub-task

  1   2   >