[jira] [Commented] (SPARK-45414) spark-xml misplaces string tag content

[ https://issues.apache.org/jira/browse/SPARK-45414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815123#comment-17815123 ] Ritika Maheshwari commented on SPARK-45414: --- So basically the fix is to replace  val

[jira] [Commented] (SPARK-45414) spark-xml misplaces string tag content

[ https://issues.apache.org/jira/browse/SPARK-45414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815121#comment-17815121 ] Ritika Maheshwari commented on SPARK-45414: --- @Sean Owen  What is causing this issue is this

[jira] [Updated] (SPARK-45414) spark-xml misplaces string tag content

[ https://issues.apache.org/jira/browse/SPARK-45414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ritika Maheshwari updated SPARK-45414: -- Attachment: Screen Shot 2024-02-01 at 7.46.29 PM.png > spark-xml misplaces string tag

[jira] [Commented] (SPARK-45414) spark-xml misplaces string tag content

[ https://issues.apache.org/jira/browse/SPARK-45414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17783841#comment-17783841 ] Ritika Maheshwari commented on SPARK-45414: --- In that case have both of these in the end like

[jira] [Commented] (SPARK-45414) spark-xml misplaces string tag content

[ https://issues.apache.org/jira/browse/SPARK-45414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17783832#comment-17783832 ] Ritika Maheshwari commented on SPARK-45414: --- Change _SerialNumberFlag to. SerialNumberFlag and

[jira] [Commented] (SPARK-40296) Error Class for DISTINCT function not found

[ https://issues.apache.org/jira/browse/SPARK-40296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744346#comment-17744346 ] Ritika Maheshwari commented on SPARK-40296: --- Isn't dropDuplicates taking care of applying

[jira] [Commented] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736945#comment-17736945 ] Ritika Maheshwari commented on SPARK-43514: --- Somehow your code is invoking the hashFunction on

[jira] [Commented] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730623#comment-17730623 ] Ritika Maheshwari commented on SPARK-43514: --- Please refer to Spark Properties section in the

[jira] [Commented] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730377#comment-17730377 ] Ritika Maheshwari commented on SPARK-43514: --- Please provide the code to recreate the issue.

[jira] [Commented] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730287#comment-17730287 ] Ritika Maheshwari commented on SPARK-43514: --- I had deleted the comment because it was

[jira] [Commented] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730261#comment-17730261 ] Ritika Maheshwari commented on SPARK-43514: --- Hello please add this to your

[jira] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

[ https://issues.apache.org/jira/browse/SPARK-43514 ] Ritika Maheshwari deleted comment on SPARK-43514: --- was (Author: ritikam): On further investigating the issue. It seems that the plan generated for 3.3.0 and 3.4.0 is same. Please see

[jira] [Commented] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730239#comment-17730239 ] Ritika Maheshwari commented on SPARK-43514: --- On further investigating the issue. It seems that

[jira] [Updated] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ritika Maheshwari updated SPARK-43514: -- Attachment: Plan.png > Unexpected NullPointerException or IllegalArgumentException

[jira] [Commented] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728230#comment-17728230 ] Ritika Maheshwari commented on SPARK-43514: --- My SPARK_HOME was still pointing to 3.3.0, once

[jira] [Updated] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ritika Maheshwari updated SPARK-43514: -- Attachment: Screen Shot 2023-05-31 at 11.14.24 PM.png > Unexpected

[jira] [Updated] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ritika Maheshwari updated SPARK-43514: -- Attachment: (was: Screen Shot 2023-05-31 at 11.04.06 PM.png) > Unexpected

[jira] [Updated] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ritika Maheshwari updated SPARK-43514: -- Attachment: (was: Screen Shot 2023-05-31 at 11.04.06 PM-1.png) > Unexpected

[jira] [Updated] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ritika Maheshwari updated SPARK-43514: -- Attachment: Screen Shot 2023-05-31 at 11.04.06 PM-1.png > Unexpected

[jira] [Updated] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ritika Maheshwari updated SPARK-43514: -- Attachment: Screen Shot 2023-05-31 at 11.04.06 PM.png > Unexpected

[jira] [Commented] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17725144#comment-17725144 ] Ritika Maheshwari commented on SPARK-43514: --- Unable to reproduce the error on 3.4.0 See the

[jira] [Updated] (SPARK-43514) Unexpected NullPointerException or IllegalArgumentException inside UDFs of ML features caused by certain SQL functions

[ https://issues.apache.org/jira/browse/SPARK-43514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ritika Maheshwari updated SPARK-43514: -- Attachment: Screen Shot 2023-05-22 at 5.39.55 PM.png > Unexpected

[jira] [Commented] (SPARK-27908) Improve parser error message for SELECT TOP statement

[ https://issues.apache.org/jira/browse/SPARK-27908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724726#comment-17724726 ] Ritika Maheshwari commented on SPARK-27908: --- In Spark 3.3.0 the error message is improved

[jira] [Commented] (SPARK-36604) timestamp type column analyze result is wrong

[ https://issues.apache.org/jira/browse/SPARK-36604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695869#comment-17695869 ] Ritika Maheshwari commented on SPARK-36604: --- Seems to be working correctly in  Spark 3.3.0

[jira] [Commented] (SPARK-40159) Aggregate should be group only after collapse project to aggregate

[ https://issues.apache.org/jira/browse/SPARK-40159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695327#comment-17695327 ] Ritika Maheshwari commented on SPARK-40159: --- This issue seems to have been resolved by

[jira] [Commented] (SPARK-30212) COUNT(DISTINCT) window function should be supported

[ https://issues.apache.org/jira/browse/SPARK-30212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17694144#comment-17694144 ] Ritika Maheshwari commented on SPARK-30212: --- This is the suggested way to handle this

[jira] [Commented] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

[ https://issues.apache.org/jira/browse/SPARK-41391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691928#comment-17691928 ] Ritika Maheshwari commented on SPARK-41391: --- I did send a PR > The output column name of

[jira] [Comment Edited] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

[ https://issues.apache.org/jira/browse/SPARK-41391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691760#comment-17691760 ] Ritika Maheshwari edited comment on SPARK-41391 at 2/21/23 8:06 PM:

[jira] [Comment Edited] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

[ https://issues.apache.org/jira/browse/SPARK-41391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691760#comment-17691760 ] Ritika Maheshwari edited comment on SPARK-41391 at 2/21/23 7:56 PM:

[jira] [Comment Edited] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

[ https://issues.apache.org/jira/browse/SPARK-41391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691760#comment-17691760 ] Ritika Maheshwari edited comment on SPARK-41391 at 2/21/23 7:35 PM:

[jira] [Comment Edited] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

[ https://issues.apache.org/jira/browse/SPARK-41391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691760#comment-17691760 ] Ritika Maheshwari edited comment on SPARK-41391 at 2/21/23 7:34 PM:

[jira] [Comment Edited] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

[ https://issues.apache.org/jira/browse/SPARK-41391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691760#comment-17691760 ] Ritika Maheshwari edited comment on SPARK-41391 at 2/21/23 7:33 PM:

[jira] [Commented] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

[ https://issues.apache.org/jira/browse/SPARK-41391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691760#comment-17691760 ] Ritika Maheshwari commented on SPARK-41391: --- A better solution   In

[jira] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

[ https://issues.apache.org/jira/browse/SPARK-41391 ] Ritika Maheshwari deleted comment on SPARK-41391: --- was (Author: ritikam): Modifying the agg method in RelationalGroupedDataset.scala to @scala.annotation.varargs def agg(expr: Column,

[jira] [Commented] (SPARK-42346) distinct(count colname) with UNION ALL causes query analyzer bug

[ https://issues.apache.org/jira/browse/SPARK-42346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17686081#comment-17686081 ] Ritika Maheshwari commented on SPARK-42346: --- Yes that caused the error to appear. Thanks >

[jira] [Commented] (SPARK-42346) distinct(count colname) with UNION ALL causes query analyzer bug

[ https://issues.apache.org/jira/browse/SPARK-42346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685466#comment-17685466 ] Ritika Maheshwari commented on SPARK-42346: --- Hello added three rows to input_table. Still no

[jira] [Commented] (SPARK-42346) distinct(count colname) with UNION ALL causes query analyzer bug

[ https://issues.apache.org/jira/browse/SPARK-42346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684992#comment-17684992 ] Ritika Maheshwari commented on SPARK-42346: --- I have Spark 3.3.0 and I do not have 39887 fix .

[jira] [Commented] (SPARK-37581) sql hang at planning stage

[ https://issues.apache.org/jira/browse/SPARK-37581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684361#comment-17684361 ] Ritika Maheshwari commented on SPARK-37581: --- Just executed this on 3.3.0  and it takes 4.14

[jira] [Commented] (SPARK-41391) The output column name of `groupBy.agg(count_distinct)` is incorrect

[ https://issues.apache.org/jira/browse/SPARK-41391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682843#comment-17682843 ] Ritika Maheshwari commented on SPARK-41391: --- Modifying the agg method in

[jira] [Commented] (SPARK-41236) The renamed field name cannot be recognized after group filtering

[ https://issues.apache.org/jira/browse/SPARK-41236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17642332#comment-17642332 ] Ritika Maheshwari commented on SPARK-41236: --- Running the following query against Spark 3.3.0

[jira] (SPARK-41236) The renamed field name cannot be recognized after group filtering

[ https://issues.apache.org/jira/browse/SPARK-41236 ] Ritika Maheshwari deleted comment on SPARK-41236: --- was (Author: ritikam): Hello Zhong , Try to rename the field as a different name than the original column name. select

[jira] [Comment Edited] (SPARK-41236) The renamed field name cannot be recognized after group filtering

[ https://issues.apache.org/jira/browse/SPARK-41236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17641576#comment-17641576 ] Ritika Maheshwari edited comment on SPARK-41236 at 11/30/22 10:51 PM:

[jira] [Commented] (SPARK-41236) The renamed field name cannot be recognized after group filtering

[ https://issues.apache.org/jira/browse/SPARK-41236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17641576#comment-17641576 ] Ritika Maheshwari commented on SPARK-41236: --- Hello Zhong , Try to rename the field as a

[jira] [Commented] (SPARK-22241) Apache spark giving InvalidSchemaException: Cannot write a schema with an empty group: optional group element {

[ https://issues.apache.org/jira/browse/SPARK-22241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16199724#comment-16199724 ] Ritika Maheshwari commented on SPARK-22241: --- I know parquet does not allow empty struct types.

[jira] [Updated] (SPARK-22241) Apache spark giving InvalidSchemaException: Cannot write a schema with an empty group: optional group element {

[ https://issues.apache.org/jira/browse/SPARK-22241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ritika Maheshwari updated SPARK-22241: -- Description: I have a bean which has field of type Arraylist of Doubles. Then I do the

[jira] [Created] (SPARK-22241) Apache spark giving InvalidSchemaException: Cannot write a schema with an empty group: optional group element {

Ritika Maheshwari created SPARK-22241: - Summary: Apache spark giving InvalidSchemaException: Cannot write a schema with an empty group: optional group element { Key: SPARK-22241 URL: