[jira] [Updated] (SPARK-44369) Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec
[ https://issues.apache.org/jira/browse/SPARK-44369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44369: - Summary: Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec (was: Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, HiveTableScanExec, SortExec) > Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec > -- > > Key: SPARK-44369 > URL: https://issues.apache.org/jira/browse/SPARK-44369 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > CollectMetricsExec > GenerateExec > ExpandExec > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44369) Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, HiveTableScanExec, SortExec
[ https://issues.apache.org/jira/browse/SPARK-44369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44369: - Description: Use PartitionEvaluator API in CollectMetricsExec GenerateExec ExpandExec was: Use PartitionEvaluator API in CollectMetricsExec GenerateExec ExpandExec HiveTableScanExec SortExec > Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, > HiveTableScanExec, SortExec > --- > > Key: SPARK-44369 > URL: https://issues.apache.org/jira/browse/SPARK-44369 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > CollectMetricsExec > GenerateExec > ExpandExec > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44359) Define the computing logic through PartitionEvaluator API and use it in BaseScriptTransformationExec, InMemoryTableScanExec, ReferenceSort, HiveTableScanExec, SortExec
[ https://issues.apache.org/jira/browse/SPARK-44359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44359: - Description: Define the computing logic through PartitionEvaluator API and use it in SQL aggregate operators BaseScriptTransformationExec InMemoryTableScanExec ReferenceSort HiveTableScanExec SortExec was: Define the computing logic through PartitionEvaluator API and use it in SQL aggregate operators BaseScriptTransformationExec InMemoryTableScanExec ReferenceSort > Define the computing logic through PartitionEvaluator API and use it in > BaseScriptTransformationExec, InMemoryTableScanExec, ReferenceSort, > HiveTableScanExec, SortExec > --- > > Key: SPARK-44359 > URL: https://issues.apache.org/jira/browse/SPARK-44359 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Define the computing logic through PartitionEvaluator API and use it in SQL > aggregate operators > BaseScriptTransformationExec > InMemoryTableScanExec > ReferenceSort > HiveTableScanExec > SortExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44359) Define the computing logic through PartitionEvaluator API and use it in BaseScriptTransformationExec, InMemoryTableScanExec, ReferenceSort, HiveTableScanExec, SortExec
[ https://issues.apache.org/jira/browse/SPARK-44359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44359: - Summary: Define the computing logic through PartitionEvaluator API and use it in BaseScriptTransformationExec, InMemoryTableScanExec, ReferenceSort, HiveTableScanExec, SortExec (was: Define the computing logic through PartitionEvaluator API and use it in BaseScriptTransformationExec, InMemoryTableScanExec, ReferenceSort) > Define the computing logic through PartitionEvaluator API and use it in > BaseScriptTransformationExec, InMemoryTableScanExec, ReferenceSort, > HiveTableScanExec, SortExec > --- > > Key: SPARK-44359 > URL: https://issues.apache.org/jira/browse/SPARK-44359 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Define the computing logic through PartitionEvaluator API and use it in SQL > aggregate operators > BaseScriptTransformationExec > InMemoryTableScanExec > ReferenceSort -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44369) Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, HiveTableScanExec, SortExec
[ https://issues.apache.org/jira/browse/SPARK-44369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44369: - Summary: Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, HiveTableScanExec, SortExec (was: Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, DebugExec, HiveTableScanExec, SortExec) > Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, > HiveTableScanExec, SortExec > --- > > Key: SPARK-44369 > URL: https://issues.apache.org/jira/browse/SPARK-44369 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > CollectMetricsExec > GenerateExec > ExpandExec > HiveTableScanExec > SortExec > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44369) Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, DebugExec, HiveTableScanExec, SortExec
[ https://issues.apache.org/jira/browse/SPARK-44369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44369: - Description: Use PartitionEvaluator API in CollectMetricsExec GenerateExec ExpandExec HiveTableScanExec SortExec was: Use PartitionEvaluator API in CollectMetricsExec GenerateExec ExpandExec DebugExec HiveTableScanExec SortExec > Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, > DebugExec, HiveTableScanExec, SortExec > -- > > Key: SPARK-44369 > URL: https://issues.apache.org/jira/browse/SPARK-44369 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > CollectMetricsExec > GenerateExec > ExpandExec > HiveTableScanExec > SortExec > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-44362) Use PartitionEvaluator API in AggregateInPandasExec, AttachDistributedSequenceExec
[ https://issues.apache.org/jira/browse/SPARK-44362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745658#comment-17745658 ] Vinod KC commented on SPARK-44362: -- PR raised : https://github.com/apache/spark/pull/42029 > Use PartitionEvaluator API in AggregateInPandasExec, > AttachDistributedSequenceExec > --- > > Key: SPARK-44362 > URL: https://issues.apache.org/jira/browse/SPARK-44362 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > `AggregateInPandasExec` > `AttachDistributedSequenceExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-44365) Use PartitionEvaluator API in FileSourceScanExec, RowDataSourceScanExec, MergeRowsExec
[ https://issues.apache.org/jira/browse/SPARK-44365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741837#comment-17741837 ] Vinod KC edited comment on SPARK-44365 at 7/21/23 4:06 PM: --- PR raised: [https://github.com/apache/spark/pull/42105] was (Author: vinodkc): Raise PR : https://github.com/apache/spark/pull/42105 > Use PartitionEvaluator API in FileSourceScanExec, RowDataSourceScanExec, > MergeRowsExec > -- > > Key: SPARK-44365 > URL: https://issues.apache.org/jira/browse/SPARK-44365 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Define the computing logic through PartitionEvaluator API and use it in SQL > operators > FileSourceScanExec > RowDataSourceScanExec > MergeRowsExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-44365) Use PartitionEvaluator API in FileSourceScanExec, RowDataSourceScanExec, MergeRowsExec
[ https://issues.apache.org/jira/browse/SPARK-44365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741837#comment-17741837 ] Vinod KC edited comment on SPARK-44365 at 7/21/23 4:15 AM: --- Raise PR : https://github.com/apache/spark/pull/42105 was (Author: vinodkc): Im working on it > Use PartitionEvaluator API in FileSourceScanExec, RowDataSourceScanExec, > MergeRowsExec > -- > > Key: SPARK-44365 > URL: https://issues.apache.org/jira/browse/SPARK-44365 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Define the computing logic through PartitionEvaluator API and use it in SQL > operators > FileSourceScanExec > RowDataSourceScanExec > MergeRowsExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44359) Define the computing logic through PartitionEvaluator API and use it in BaseScriptTransformationExec, InMemoryTableScanExec, ReferenceSort
[ https://issues.apache.org/jira/browse/SPARK-44359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44359: - Summary: Define the computing logic through PartitionEvaluator API and use it in BaseScriptTransformationExec, InMemoryTableScanExec, ReferenceSort (was: Define the computing logic through PartitionEvaluator API and use it in BaseScriptTransformationExec) > Define the computing logic through PartitionEvaluator API and use it in > BaseScriptTransformationExec, InMemoryTableScanExec, ReferenceSort > -- > > Key: SPARK-44359 > URL: https://issues.apache.org/jira/browse/SPARK-44359 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Define the computing logic through PartitionEvaluator API and use it in SQL > aggregate operators > BaseScriptTransformationExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44359) Define the computing logic through PartitionEvaluator API and use it in BaseScriptTransformationExec, InMemoryTableScanExec, ReferenceSort
[ https://issues.apache.org/jira/browse/SPARK-44359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44359: - Description: Define the computing logic through PartitionEvaluator API and use it in SQL aggregate operators BaseScriptTransformationExec InMemoryTableScanExec ReferenceSort was: Define the computing logic through PartitionEvaluator API and use it in SQL aggregate operators BaseScriptTransformationExec > Define the computing logic through PartitionEvaluator API and use it in > BaseScriptTransformationExec, InMemoryTableScanExec, ReferenceSort > -- > > Key: SPARK-44359 > URL: https://issues.apache.org/jira/browse/SPARK-44359 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Define the computing logic through PartitionEvaluator API and use it in SQL > aggregate operators > BaseScriptTransformationExec > InMemoryTableScanExec > ReferenceSort -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44365) Use PartitionEvaluator API in FileSourceScanExec, RowDataSourceScanExec, MergeRowsExec
[ https://issues.apache.org/jira/browse/SPARK-44365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44365: - Description: Define the computing logic through PartitionEvaluator API and use it in SQL operators FileSourceScanExec RowDataSourceScanExec MergeRowsExec was: Define the computing logic through PartitionEvaluator API and use it in SQL operators InMemoryTableScanExec DataSourceScanExec MergeRowsExec ReferenceSort > Use PartitionEvaluator API in FileSourceScanExec, RowDataSourceScanExec, > MergeRowsExec > -- > > Key: SPARK-44365 > URL: https://issues.apache.org/jira/browse/SPARK-44365 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Define the computing logic through PartitionEvaluator API and use it in SQL > operators > FileSourceScanExec > RowDataSourceScanExec > MergeRowsExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44365) Use PartitionEvaluator API in FileSourceScanExec, RowDataSourceScanExec, MergeRowsExec
[ https://issues.apache.org/jira/browse/SPARK-44365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44365: - Summary: Use PartitionEvaluator API in FileSourceScanExec, RowDataSourceScanExec, MergeRowsExec (was: Define the computing logic through PartitionEvaluator API and use it in SQL operators InMemoryTableScanExec, DataSourceScanExec, MergeRowsExec , ReferenceSort) > Use PartitionEvaluator API in FileSourceScanExec, RowDataSourceScanExec, > MergeRowsExec > -- > > Key: SPARK-44365 > URL: https://issues.apache.org/jira/browse/SPARK-44365 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Define the computing logic through PartitionEvaluator API and use it in SQL > operators > InMemoryTableScanExec > DataSourceScanExec > MergeRowsExec > ReferenceSort -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44362) Use PartitionEvaluator API in AggregateInPandasExec, AttachDistributedSequenceExec
[ https://issues.apache.org/jira/browse/SPARK-44362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44362: - Description: Use PartitionEvaluator API in `AggregateInPandasExec` `AttachDistributedSequenceExec` was: Use PartitionEvaluator API in AggregateInPandasExec AttachDistributedSequenceExec DeserializeToObjectExec > Use PartitionEvaluator API in AggregateInPandasExec, > AttachDistributedSequenceExec > --- > > Key: SPARK-44362 > URL: https://issues.apache.org/jira/browse/SPARK-44362 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > `AggregateInPandasExec` > `AttachDistributedSequenceExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44362) Use PartitionEvaluator API in AggregateInPandasExec, AttachDistributedSequenceExec
[ https://issues.apache.org/jira/browse/SPARK-44362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44362: - Summary: Use PartitionEvaluator API in AggregateInPandasExec, AttachDistributedSequenceExec (was: Use PartitionEvaluator API in AggregateInPandasExec, AttachDistributedSequenceExec, DeserializeToObjectExec) > Use PartitionEvaluator API in AggregateInPandasExec, > AttachDistributedSequenceExec > --- > > Key: SPARK-44362 > URL: https://issues.apache.org/jira/browse/SPARK-44362 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > AggregateInPandasExec > AttachDistributedSequenceExec > DeserializeToObjectExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-44447) Use PartitionEvaluator API in FlatMapGroupsInPandasExec, FlatMapCoGroupsInPandasExec
[ https://issues.apache.org/jira/browse/SPARK-7?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17743572#comment-17743572 ] Vinod KC edited comment on SPARK-7 at 7/16/23 9:56 PM: --- Raised PR : https://github.com/apache/spark/pull/42025 was (Author: vinodkc): I'm working on it > Use PartitionEvaluator API in FlatMapGroupsInPandasExec, > FlatMapCoGroupsInPandasExec > > > Key: SPARK-7 > URL: https://issues.apache.org/jira/browse/SPARK-7 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > `FlatMapGroupsInPandasExec` > `FlatMapCoGroupsInPandasExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-44361) Use PartitionEvaluator API in MapInBatchExec
[ https://issues.apache.org/jira/browse/SPARK-44361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741836#comment-17741836 ] Vinod KC edited comment on SPARK-44361 at 7/16/23 9:48 PM: --- PR raised : https://github.com/apache/spark/pull/42024 was (Author: vinodkc): Im working on it > Use PartitionEvaluator API in MapInBatchExec > - > > Key: SPARK-44361 > URL: https://issues.apache.org/jira/browse/SPARK-44361 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > `MapInBatchExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-44447) Use PartitionEvaluator API in FlatMapGroupsInPandasExec, FlatMapCoGroupsInPandasExec
Vinod KC created SPARK-7: Summary: Use PartitionEvaluator API in FlatMapGroupsInPandasExec, FlatMapCoGroupsInPandasExec Key: SPARK-7 URL: https://issues.apache.org/jira/browse/SPARK-7 Project: Spark Issue Type: Sub-task Components: SQL Affects Versions: 3.5.0 Reporter: Vinod KC Use PartitionEvaluator API in `FlatMapGroupsInPandasExec` `FlatMapCoGroupsInPandasExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-44447) Use PartitionEvaluator API in FlatMapGroupsInPandasExec, FlatMapCoGroupsInPandasExec
[ https://issues.apache.org/jira/browse/SPARK-7?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17743572#comment-17743572 ] Vinod KC commented on SPARK-7: -- I'm working on it > Use PartitionEvaluator API in FlatMapGroupsInPandasExec, > FlatMapCoGroupsInPandasExec > > > Key: SPARK-7 > URL: https://issues.apache.org/jira/browse/SPARK-7 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > `FlatMapGroupsInPandasExec` > `FlatMapCoGroupsInPandasExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44361) Use PartitionEvaluator API in MapInBatchExec
[ https://issues.apache.org/jira/browse/SPARK-44361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44361: - Summary: Use PartitionEvaluator API in MapInBatchExec (was: Use PartitionEvaluator API in FlatMapGroupsInPandasExec, MapInBatchExec, FlatMapCoGroupsInPandasExec) > Use PartitionEvaluator API in MapInBatchExec > - > > Key: SPARK-44361 > URL: https://issues.apache.org/jira/browse/SPARK-44361 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > `FlatMapGroupsInPandasExec` > `MapInBatchExec` > `FlatMapCoGroupsInPandasExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44361) Use PartitionEvaluator API in MapInBatchExec
[ https://issues.apache.org/jira/browse/SPARK-44361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44361: - Description: Use PartitionEvaluator API in `MapInBatchExec` was: Use PartitionEvaluator API in `FlatMapGroupsInPandasExec` `MapInBatchExec` `FlatMapCoGroupsInPandasExec` > Use PartitionEvaluator API in MapInBatchExec > - > > Key: SPARK-44361 > URL: https://issues.apache.org/jira/browse/SPARK-44361 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > `MapInBatchExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] (SPARK-44412) Use PartitionEvaluator API in ArrowEvalPythonUDTFExec & BatchEvalPythonUDTFExec
[ https://issues.apache.org/jira/browse/SPARK-44412 ] Vinod KC deleted comment on SPARK-44412: -- was (Author: vinodkc): I'm working on this > Use PartitionEvaluator API in ArrowEvalPythonUDTFExec & > BatchEvalPythonUDTFExec > --- > > Key: SPARK-44412 > URL: https://issues.apache.org/jira/browse/SPARK-44412 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > `ArrowEvalPythonUDTFExec` > `BatchEvalPythonUDTFExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-44412) Use PartitionEvaluator API in ArrowEvalPythonUDTFExec & BatchEvalPythonUDTFExec
[ https://issues.apache.org/jira/browse/SPARK-44412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17743335#comment-17743335 ] Vinod KC commented on SPARK-44412: -- Raised PR : https://github.com/apache/spark/pull/42014 > Use PartitionEvaluator API in ArrowEvalPythonUDTFExec & > BatchEvalPythonUDTFExec > --- > > Key: SPARK-44412 > URL: https://issues.apache.org/jira/browse/SPARK-44412 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > `ArrowEvalPythonUDTFExec` > `BatchEvalPythonUDTFExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44412) Use PartitionEvaluator API in ArrowEvalPythonUDTFExec & BatchEvalPythonUDTFExec
[ https://issues.apache.org/jira/browse/SPARK-44412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44412: - Description: Use PartitionEvaluator API in `ArrowEvalPythonUDTFExec` `BatchEvalPythonUDTFExec` was: Use PartitionEvaluator API in `EvalPythonUDTFExec` `ArrowEvalPythonUDTFExec` `BatchEvalPythonUDTFExec` > Use PartitionEvaluator API in ArrowEvalPythonUDTFExec & > BatchEvalPythonUDTFExec > --- > > Key: SPARK-44412 > URL: https://issues.apache.org/jira/browse/SPARK-44412 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > `ArrowEvalPythonUDTFExec` > `BatchEvalPythonUDTFExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44412) Use PartitionEvaluator API in ArrowEvalPythonUDTFExec & BatchEvalPythonUDTFExec
[ https://issues.apache.org/jira/browse/SPARK-44412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44412: - Summary: Use PartitionEvaluator API in ArrowEvalPythonUDTFExec & BatchEvalPythonUDTFExec (was: Use PartitionEvaluator API in EvalPythonUDTFExec,ArrowEvalPythonUDTFExec, BatchEvalPythonUDTFExec) > Use PartitionEvaluator API in ArrowEvalPythonUDTFExec & > BatchEvalPythonUDTFExec > --- > > Key: SPARK-44412 > URL: https://issues.apache.org/jira/browse/SPARK-44412 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > `EvalPythonUDTFExec` > `ArrowEvalPythonUDTFExec` > `BatchEvalPythonUDTFExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44411) Use PartitionEvaluator API in ArrowEvalPythonExec, BatchEvalPythonExec
[ https://issues.apache.org/jira/browse/SPARK-44411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44411: - Description: Use PartitionEvaluator API in `ArrowEvalPythonExec` `BatchEvalPythonExec` was: Use PartitionEvaluator API in `EvalPythonExec` `ArrowEvalPythonExec` `BatchEvalPythonExec` > Use PartitionEvaluator API in ArrowEvalPythonExec, BatchEvalPythonExec > -- > > Key: SPARK-44411 > URL: https://issues.apache.org/jira/browse/SPARK-44411 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > `ArrowEvalPythonExec` > `BatchEvalPythonExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-44411) Use PartitionEvaluator API in EvalPythonExec,ArrowEvalPythonExec, BatchEvalPythonExec
[ https://issues.apache.org/jira/browse/SPARK-44411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17743011#comment-17743011 ] Vinod KC commented on SPARK-44411: -- PR raised : https://github.com/apache/spark/pull/41998 > Use PartitionEvaluator API in EvalPythonExec,ArrowEvalPythonExec, > BatchEvalPythonExec > - > > Key: SPARK-44411 > URL: https://issues.apache.org/jira/browse/SPARK-44411 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > `EvalPythonExec` > `ArrowEvalPythonExec` > `BatchEvalPythonExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44411) Use PartitionEvaluator API in ArrowEvalPythonExec, BatchEvalPythonExec
[ https://issues.apache.org/jira/browse/SPARK-44411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44411: - Summary: Use PartitionEvaluator API in ArrowEvalPythonExec, BatchEvalPythonExec (was: Use PartitionEvaluator API in EvalPythonExec,ArrowEvalPythonExec, BatchEvalPythonExec) > Use PartitionEvaluator API in ArrowEvalPythonExec, BatchEvalPythonExec > -- > > Key: SPARK-44411 > URL: https://issues.apache.org/jira/browse/SPARK-44411 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > `EvalPythonExec` > `ArrowEvalPythonExec` > `BatchEvalPythonExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-44412) Use PartitionEvaluator API in EvalPythonUDTFExec,ArrowEvalPythonUDTFExec, BatchEvalPythonUDTFExec
[ https://issues.apache.org/jira/browse/SPARK-44412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742965#comment-17742965 ] Vinod KC commented on SPARK-44412: -- I'm working on this > Use PartitionEvaluator API in EvalPythonUDTFExec,ArrowEvalPythonUDTFExec, > BatchEvalPythonUDTFExec > - > > Key: SPARK-44412 > URL: https://issues.apache.org/jira/browse/SPARK-44412 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > `EvalPythonUDTFExec` > `ArrowEvalPythonUDTFExec` > `BatchEvalPythonUDTFExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44361) Use PartitionEvaluator API in FlatMapGroupsInPandasExec, MapInBatchExec, FlatMapCoGroupsInPandasExec
[ https://issues.apache.org/jira/browse/SPARK-44361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44361: - Description: Use PartitionEvaluator API in `FlatMapGroupsInPandasExec` `MapInBatchExec` `FlatMapCoGroupsInPandasExec` was: Use PartitionEvaluator API in EvalPythonUDTFExec BatchEvalPythonUDTFExec, FlatMapGroupsInPandasExec, MapInBatchExec, FlatMapCoGroupsInPandasExec > Use PartitionEvaluator API in FlatMapGroupsInPandasExec, MapInBatchExec, > FlatMapCoGroupsInPandasExec > - > > Key: SPARK-44361 > URL: https://issues.apache.org/jira/browse/SPARK-44361 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > `FlatMapGroupsInPandasExec` > `MapInBatchExec` > `FlatMapCoGroupsInPandasExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44361) Use PartitionEvaluator API in FlatMapGroupsInPandasExec, MapInBatchExec, FlatMapCoGroupsInPandasExec
[ https://issues.apache.org/jira/browse/SPARK-44361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44361: - Summary: Use PartitionEvaluator API in FlatMapGroupsInPandasExec, MapInBatchExec, FlatMapCoGroupsInPandasExec (was: Use PartitionEvaluator API in EvalPythonUDTFExec, BatchEvalPythonUDTFExec, FlatMapGroupsInPandasExec, MapInBatchExec, FlatMapCoGroupsInPandasExec) > Use PartitionEvaluator API in FlatMapGroupsInPandasExec, MapInBatchExec, > FlatMapCoGroupsInPandasExec > - > > Key: SPARK-44361 > URL: https://issues.apache.org/jira/browse/SPARK-44361 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > EvalPythonUDTFExec > BatchEvalPythonUDTFExec, > FlatMapGroupsInPandasExec, > MapInBatchExec, > FlatMapCoGroupsInPandasExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-44412) Use PartitionEvaluator API in EvalPythonUDTFExec,ArrowEvalPythonUDTFExec, BatchEvalPythonUDTFExec
Vinod KC created SPARK-44412: Summary: Use PartitionEvaluator API in EvalPythonUDTFExec,ArrowEvalPythonUDTFExec, BatchEvalPythonUDTFExec Key: SPARK-44412 URL: https://issues.apache.org/jira/browse/SPARK-44412 Project: Spark Issue Type: Sub-task Components: SQL Affects Versions: 3.5.0 Reporter: Vinod KC Use PartitionEvaluator API in `EvalPythonUDTFExec` `ArrowEvalPythonUDTFExec` `BatchEvalPythonUDTFExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44361) Use PartitionEvaluator API in BatchEvalPythonUDTFExec, FlatMapGroupsInPandasExec, MapInBatchExec, FlatMapCoGroupsInPandasExec
[ https://issues.apache.org/jira/browse/SPARK-44361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44361: - Description: Use PartitionEvaluator API in EvalPythonUDTFExec BatchEvalPythonUDTFExec, FlatMapGroupsInPandasExec, MapInBatchExec, FlatMapCoGroupsInPandasExec was: Use PartitionEvaluator API in BatchEvalPythonUDTFExec, FlatMapGroupsInPandasExec, MapInBatchExec, FlatMapCoGroupsInPandasExec > Use PartitionEvaluator API in BatchEvalPythonUDTFExec, > FlatMapGroupsInPandasExec, MapInBatchExec, FlatMapCoGroupsInPandasExec > -- > > Key: SPARK-44361 > URL: https://issues.apache.org/jira/browse/SPARK-44361 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > EvalPythonUDTFExec > BatchEvalPythonUDTFExec, > FlatMapGroupsInPandasExec, > MapInBatchExec, > FlatMapCoGroupsInPandasExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44361) Use PartitionEvaluator API in EvalPythonUDTFExec, BatchEvalPythonUDTFExec, FlatMapGroupsInPandasExec, MapInBatchExec, FlatMapCoGroupsInPandasExec
[ https://issues.apache.org/jira/browse/SPARK-44361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44361: - Summary: Use PartitionEvaluator API in EvalPythonUDTFExec, BatchEvalPythonUDTFExec, FlatMapGroupsInPandasExec, MapInBatchExec, FlatMapCoGroupsInPandasExec (was: Use PartitionEvaluator API in BatchEvalPythonUDTFExec, FlatMapGroupsInPandasExec, MapInBatchExec, FlatMapCoGroupsInPandasExec) > Use PartitionEvaluator API in EvalPythonUDTFExec, BatchEvalPythonUDTFExec, > FlatMapGroupsInPandasExec, MapInBatchExec, FlatMapCoGroupsInPandasExec > -- > > Key: SPARK-44361 > URL: https://issues.apache.org/jira/browse/SPARK-44361 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > EvalPythonUDTFExec > BatchEvalPythonUDTFExec, > FlatMapGroupsInPandasExec, > MapInBatchExec, > FlatMapCoGroupsInPandasExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44362) Use PartitionEvaluator API in AggregateInPandasExec, AttachDistributedSequenceExec, DeserializeToObjectExec
[ https://issues.apache.org/jira/browse/SPARK-44362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44362: - Summary: Use PartitionEvaluator API in AggregateInPandasExec, AttachDistributedSequenceExec, DeserializeToObjectExec (was: Use PartitionEvaluator API in AggregateInPandasExec,EvalPythonExec,AttachDistributedSequenceExec, DeserializeToObjectExec) > Use PartitionEvaluator API in AggregateInPandasExec, > AttachDistributedSequenceExec, DeserializeToObjectExec > > > Key: SPARK-44362 > URL: https://issues.apache.org/jira/browse/SPARK-44362 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > AggregateInPandasExec > AttachDistributedSequenceExec > DeserializeToObjectExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44362) Use PartitionEvaluator API in AggregateInPandasExec,EvalPythonExec,AttachDistributedSequenceExec, DeserializeToObjectExec
[ https://issues.apache.org/jira/browse/SPARK-44362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44362: - Description: Use PartitionEvaluator API in AggregateInPandasExec AttachDistributedSequenceExec DeserializeToObjectExec was: Use PartitionEvaluator API in AggregateInPandasExec EvalPythonExec AttachDistributedSequenceExec DeserializeToObjectExec > Use PartitionEvaluator API in > AggregateInPandasExec,EvalPythonExec,AttachDistributedSequenceExec, > DeserializeToObjectExec > -- > > Key: SPARK-44362 > URL: https://issues.apache.org/jira/browse/SPARK-44362 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > AggregateInPandasExec > AttachDistributedSequenceExec > DeserializeToObjectExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-44411) Use PartitionEvaluator API in EvalPythonExec,ArrowEvalPythonExec, BatchEvalPythonExec
[ https://issues.apache.org/jira/browse/SPARK-44411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742944#comment-17742944 ] Vinod KC commented on SPARK-44411: -- I'm working on this > Use PartitionEvaluator API in EvalPythonExec,ArrowEvalPythonExec, > BatchEvalPythonExec > - > > Key: SPARK-44411 > URL: https://issues.apache.org/jira/browse/SPARK-44411 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > `EvalPythonExec` > `ArrowEvalPythonExec` > `BatchEvalPythonExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-44411) Use PartitionEvaluator API in EvalPythonExec,ArrowEvalPythonExec, BatchEvalPythonExec
Vinod KC created SPARK-44411: Summary: Use PartitionEvaluator API in EvalPythonExec,ArrowEvalPythonExec, BatchEvalPythonExec Key: SPARK-44411 URL: https://issues.apache.org/jira/browse/SPARK-44411 Project: Spark Issue Type: Sub-task Components: SQL Affects Versions: 3.5.0 Reporter: Vinod KC Use PartitionEvaluator API in `EvalPythonExec` `ArrowEvalPythonExec` `BatchEvalPythonExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44359) Define the computing logic through PartitionEvaluator API and use it in BaseScriptTransformationExec
[ https://issues.apache.org/jira/browse/SPARK-44359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44359: - Summary: Define the computing logic through PartitionEvaluator API and use it in BaseScriptTransformationExec (was: Define the computing logic through PartitionEvaluator API and use it in SQL aggregate operators) > Define the computing logic through PartitionEvaluator API and use it in > BaseScriptTransformationExec > > > Key: SPARK-44359 > URL: https://issues.apache.org/jira/browse/SPARK-44359 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Define the computing logic through PartitionEvaluator API and use it in SQL > aggregate operators > BaseScriptTransformationExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44359) Define the computing logic through PartitionEvaluator API and use it in SQL aggregate operators
[ https://issues.apache.org/jira/browse/SPARK-44359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44359: - Description: Define the computing logic through PartitionEvaluator API and use it in SQL aggregate operators BaseScriptTransformationExec was: Define the computing logic through PartitionEvaluator API and use it in SQL aggregate operators `MergingSessionsExec` `SortAggregateExec` `UpdatingSessionsExec` `HashAggregateExec` `ObjectHashAggregateExec` > Define the computing logic through PartitionEvaluator API and use it in SQL > aggregate operators > --- > > Key: SPARK-44359 > URL: https://issues.apache.org/jira/browse/SPARK-44359 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Define the computing logic through PartitionEvaluator API and use it in SQL > aggregate operators > BaseScriptTransformationExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44362) Use PartitionEvaluator API in AggregateInPandasExec,EvalPythonExec,AttachDistributedSequenceExec, DeserializeToObjectExec
[ https://issues.apache.org/jira/browse/SPARK-44362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44362: - Description: Use PartitionEvaluator API in AggregateInPandasExec EvalPythonExec AttachDistributedSequenceExec DeserializeToObjectExec was: Use PartitionEvaluator API in AggregateInPandasExec EvalPythonExec AttachDistributedSequenceExec > Use PartitionEvaluator API in > AggregateInPandasExec,EvalPythonExec,AttachDistributedSequenceExec, > DeserializeToObjectExec > -- > > Key: SPARK-44362 > URL: https://issues.apache.org/jira/browse/SPARK-44362 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > AggregateInPandasExec > EvalPythonExec > AttachDistributedSequenceExec > DeserializeToObjectExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44362) Use PartitionEvaluator API in AggregateInPandasExec,EvalPythonExec,AttachDistributedSequenceExec, DeserializeToObjectExec
[ https://issues.apache.org/jira/browse/SPARK-44362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44362: - Summary: Use PartitionEvaluator API in AggregateInPandasExec,EvalPythonExec,AttachDistributedSequenceExec, DeserializeToObjectExec (was: Use PartitionEvaluator API in AggregateInPandasExec,EvalPythonExec,AttachDistributedSequenceExec) > Use PartitionEvaluator API in > AggregateInPandasExec,EvalPythonExec,AttachDistributedSequenceExec, > DeserializeToObjectExec > -- > > Key: SPARK-44362 > URL: https://issues.apache.org/jira/browse/SPARK-44362 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > AggregateInPandasExec > EvalPythonExec > AttachDistributedSequenceExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44369) Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, DebugExec, HiveTableScanExec, SortExec
[ https://issues.apache.org/jira/browse/SPARK-44369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44369: - Summary: Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, DebugExec, HiveTableScanExec, SortExec (was: Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, DebugExec, HiveTableScanExec, DataSourceScanExec, SortExec) > Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, > DebugExec, HiveTableScanExec, SortExec > -- > > Key: SPARK-44369 > URL: https://issues.apache.org/jira/browse/SPARK-44369 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > CollectMetricsExec > GenerateExec > ExpandExec > DebugExec > HiveTableScanExec > SortExec > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44369) Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, DebugExec, HiveTableScanExec, DataSourceScanExec, SortExec
[ https://issues.apache.org/jira/browse/SPARK-44369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44369: - Description: Use PartitionEvaluator API in CollectMetricsExec GenerateExec ExpandExec DebugExec HiveTableScanExec SortExec was: Use PartitionEvaluator API in CollectMetricsExec GenerateExec ExpandExec DebugExec HiveTableScanExec DataSourceScanExec SortExec > Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, > DebugExec, HiveTableScanExec, DataSourceScanExec, SortExec > -- > > Key: SPARK-44369 > URL: https://issues.apache.org/jira/browse/SPARK-44369 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > CollectMetricsExec > GenerateExec > ExpandExec > DebugExec > HiveTableScanExec > SortExec > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44362) Use PartitionEvaluator API in AggregateInPandasExec,EvalPythonExec,AttachDistributedSequenceExec
[ https://issues.apache.org/jira/browse/SPARK-44362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44362: - Summary: Use PartitionEvaluator API in AggregateInPandasExec,EvalPythonExec,AttachDistributedSequenceExec (was: Use PartitionEvaluator API in AggregateInPandasExec, WindowInPandasExec,EvalPythonExec,AttachDistributedSequenceExec) > Use PartitionEvaluator API in > AggregateInPandasExec,EvalPythonExec,AttachDistributedSequenceExec > - > > Key: SPARK-44362 > URL: https://issues.apache.org/jira/browse/SPARK-44362 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > AggregateInPandasExec > EvalPythonExec > AttachDistributedSequenceExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44362) Use PartitionEvaluator API in AggregateInPandasExec, WindowInPandasExec,EvalPythonExec,AttachDistributedSequenceExec
[ https://issues.apache.org/jira/browse/SPARK-44362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44362: - Description: Use PartitionEvaluator API in AggregateInPandasExec EvalPythonExec AttachDistributedSequenceExec was: Use PartitionEvaluator API in AggregateInPandasExec WindowInPandasExec EvalPythonExec AttachDistributedSequenceExec > Use PartitionEvaluator API in AggregateInPandasExec, > WindowInPandasExec,EvalPythonExec,AttachDistributedSequenceExec > - > > Key: SPARK-44362 > URL: https://issues.apache.org/jira/browse/SPARK-44362 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > AggregateInPandasExec > EvalPythonExec > AttachDistributedSequenceExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-44362) Use PartitionEvaluator API in AggregateInPandasExec, WindowInPandasExec,EvalPythonExec,AttachDistributedSequenceExec
[ https://issues.apache.org/jira/browse/SPARK-44362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742099#comment-17742099 ] Vinod KC commented on SPARK-44362: -- yes, please go ahead > Use PartitionEvaluator API in AggregateInPandasExec, > WindowInPandasExec,EvalPythonExec,AttachDistributedSequenceExec > - > > Key: SPARK-44362 > URL: https://issues.apache.org/jira/browse/SPARK-44362 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > AggregateInPandasExec > WindowInPandasExec > EvalPythonExec > AttachDistributedSequenceExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44365) Define the computing logic through PartitionEvaluator API and use it in SQL operators InMemoryTableScanExec, DataSourceScanExec, MergeRowsExec , ReferenceSort
[ https://issues.apache.org/jira/browse/SPARK-44365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44365: - Description: Define the computing logic through PartitionEvaluator API and use it in SQL operators InMemoryTableScanExec DataSourceScanExec MergeRowsExec ReferenceSort was:Define the computing logic through PartitionEvaluator API and use it in SQL operators `InMemoryTableScanExec` > Define the computing logic through PartitionEvaluator API and use it in SQL > operators InMemoryTableScanExec, DataSourceScanExec, MergeRowsExec , > ReferenceSort > -- > > Key: SPARK-44365 > URL: https://issues.apache.org/jira/browse/SPARK-44365 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Define the computing logic through PartitionEvaluator API and use it in SQL > operators > InMemoryTableScanExec > DataSourceScanExec > MergeRowsExec > ReferenceSort -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44365) Define the computing logic through PartitionEvaluator API and use it in SQL operators InMemoryTableScanExec, DataSourceScanExec, MergeRowsExec , ReferenceSort
[ https://issues.apache.org/jira/browse/SPARK-44365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44365: - Summary: Define the computing logic through PartitionEvaluator API and use it in SQL operators InMemoryTableScanExec, DataSourceScanExec, MergeRowsExec , ReferenceSort (was: Define the computing logic through PartitionEvaluator API and use it in SQL operators InMemoryTableScanExec) > Define the computing logic through PartitionEvaluator API and use it in SQL > operators InMemoryTableScanExec, DataSourceScanExec, MergeRowsExec , > ReferenceSort > -- > > Key: SPARK-44365 > URL: https://issues.apache.org/jira/browse/SPARK-44365 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Define the computing logic through PartitionEvaluator API and use it in SQL > operators `InMemoryTableScanExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44369) Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, DebugExec, HiveTableScanExec, DataSourceScanExec, SortExec
[ https://issues.apache.org/jira/browse/SPARK-44369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44369: - Summary: Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, DebugExec, HiveTableScanExec, DataSourceScanExec, SortExec (was: Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, DebugExec, HiveTableScanExec, DataSourceScanExec) > Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, > DebugExec, HiveTableScanExec, DataSourceScanExec, SortExec > -- > > Key: SPARK-44369 > URL: https://issues.apache.org/jira/browse/SPARK-44369 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > CollectMetricsExec > GenerateExec > ExpandExec > DebugExec > HiveTableScanExec > DataSourceScanExec > SortExec > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44369) Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, DebugExec, HiveTableScanExec, DataSourceScanExec
[ https://issues.apache.org/jira/browse/SPARK-44369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44369: - Description: Use PartitionEvaluator API in CollectMetricsExec GenerateExec ExpandExec DebugExec HiveTableScanExec DataSourceScanExec SortExec was: Use PartitionEvaluator API in CollectMetricsExec GenerateExec ExpandExec DebugExec HiveTableScanExec DataSourceScanExec > Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, > DebugExec, HiveTableScanExec, DataSourceScanExec > > > Key: SPARK-44369 > URL: https://issues.apache.org/jira/browse/SPARK-44369 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > CollectMetricsExec > GenerateExec > ExpandExec > DebugExec > HiveTableScanExec > DataSourceScanExec > SortExec > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-44369) Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, DebugExec, HiveTableScanExec, DataSourceScanExec
[ https://issues.apache.org/jira/browse/SPARK-44369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741839#comment-17741839 ] Vinod KC commented on SPARK-44369: -- I'm working on it > Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, > DebugExec, HiveTableScanExec, DataSourceScanExec > > > Key: SPARK-44369 > URL: https://issues.apache.org/jira/browse/SPARK-44369 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > CollectMetricsExec > GenerateExec > ExpandExec > DebugExec > HiveTableScanExec > DataSourceScanExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-44369) Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, DebugExec, HiveTableScanExec, DataSourceScanExec
Vinod KC created SPARK-44369: Summary: Use PartitionEvaluator API in CollectMetricsExec, GenerateExec, ExpandExec, DebugExec, HiveTableScanExec, DataSourceScanExec Key: SPARK-44369 URL: https://issues.apache.org/jira/browse/SPARK-44369 Project: Spark Issue Type: Sub-task Components: SQL Affects Versions: 3.5.0 Reporter: Vinod KC Use PartitionEvaluator API in CollectMetricsExec GenerateExec ExpandExec DebugExec HiveTableScanExec DataSourceScanExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-44362) Use PartitionEvaluator API in AggregateInPandasExec, WindowInPandasExec,EvalPythonExec,AttachDistributedSequenceExec
[ https://issues.apache.org/jira/browse/SPARK-44362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741838#comment-17741838 ] Vinod KC commented on SPARK-44362: -- Im working on it > Use PartitionEvaluator API in AggregateInPandasExec, > WindowInPandasExec,EvalPythonExec,AttachDistributedSequenceExec > - > > Key: SPARK-44362 > URL: https://issues.apache.org/jira/browse/SPARK-44362 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > AggregateInPandasExec > WindowInPandasExec > EvalPythonExec > AttachDistributedSequenceExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-44365) Define the computing logic through PartitionEvaluator API and use it in SQL operators InMemoryTableScanExec
[ https://issues.apache.org/jira/browse/SPARK-44365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741837#comment-17741837 ] Vinod KC commented on SPARK-44365: -- Im working on it > Define the computing logic through PartitionEvaluator API and use it in SQL > operators InMemoryTableScanExec > --- > > Key: SPARK-44365 > URL: https://issues.apache.org/jira/browse/SPARK-44365 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Define the computing logic through PartitionEvaluator API and use it in SQL > operators `InMemoryTableScanExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-44361) Use PartitionEvaluator API in BatchEvalPythonUDTFExec, FlatMapGroupsInPandasExec, MapInBatchExec, FlatMapCoGroupsInPandasExec
[ https://issues.apache.org/jira/browse/SPARK-44361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741836#comment-17741836 ] Vinod KC commented on SPARK-44361: -- Im working on it > Use PartitionEvaluator API in BatchEvalPythonUDTFExec, > FlatMapGroupsInPandasExec, MapInBatchExec, FlatMapCoGroupsInPandasExec > -- > > Key: SPARK-44361 > URL: https://issues.apache.org/jira/browse/SPARK-44361 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Use PartitionEvaluator API in > BatchEvalPythonUDTFExec, > FlatMapGroupsInPandasExec, > MapInBatchExec, > FlatMapCoGroupsInPandasExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44365) Use PartitionEvaluator API in AggregateInPandasExec, WindowInPandasExec,EvalPythonExec,AttachDistributedSequenceExec
[ https://issues.apache.org/jira/browse/SPARK-44365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44365: - Description: Define the computing logic through PartitionEvaluator API and use it in SQL operators `InMemoryTableScanExec` (was: InMemoryTableScanExec) > Use PartitionEvaluator API in AggregateInPandasExec, > WindowInPandasExec,EvalPythonExec,AttachDistributedSequenceExec > - > > Key: SPARK-44365 > URL: https://issues.apache.org/jira/browse/SPARK-44365 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Define the computing logic through PartitionEvaluator API and use it in SQL > operators `InMemoryTableScanExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44365) Use PartitionEvaluator API in AggregateInPandasExec, WindowInPandasExec,EvalPythonExec,AttachDistributedSequenceExec
[ https://issues.apache.org/jira/browse/SPARK-44365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44365: - Description: InMemoryTableScanExec (was: `BatchEvalPythonUDTFExec` `FlatMapGroupsInPandasExec` `MapInBatchExec` `FlatMapCoGroupsInPandasExec`) > Use PartitionEvaluator API in AggregateInPandasExec, > WindowInPandasExec,EvalPythonExec,AttachDistributedSequenceExec > - > > Key: SPARK-44365 > URL: https://issues.apache.org/jira/browse/SPARK-44365 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > InMemoryTableScanExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44365) Define the computing logic through PartitionEvaluator API and use it in SQL operators InMemoryTableScanExec
[ https://issues.apache.org/jira/browse/SPARK-44365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44365: - Summary: Define the computing logic through PartitionEvaluator API and use it in SQL operators InMemoryTableScanExec (was: Use PartitionEvaluator API in AggregateInPandasExec, WindowInPandasExec,EvalPythonExec,AttachDistributedSequenceExec) > Define the computing logic through PartitionEvaluator API and use it in SQL > operators InMemoryTableScanExec > --- > > Key: SPARK-44365 > URL: https://issues.apache.org/jira/browse/SPARK-44365 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Define the computing logic through PartitionEvaluator API and use it in SQL > operators `InMemoryTableScanExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-44365) Use PartitionEvaluator API in AggregateInPandasExec, WindowInPandasExec,EvalPythonExec,AttachDistributedSequenceExec
Vinod KC created SPARK-44365: Summary: Use PartitionEvaluator API in AggregateInPandasExec, WindowInPandasExec,EvalPythonExec,AttachDistributedSequenceExec Key: SPARK-44365 URL: https://issues.apache.org/jira/browse/SPARK-44365 Project: Spark Issue Type: Sub-task Components: SQL Affects Versions: 3.5.0 Reporter: Vinod KC `BatchEvalPythonUDTFExec` `FlatMapGroupsInPandasExec` `MapInBatchExec` `FlatMapCoGroupsInPandasExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-44362) Use PartitionEvaluator API in AggregateInPandasExec, WindowInPandasExec,EvalPythonExec,AttachDistributedSequenceExec
Vinod KC created SPARK-44362: Summary: Use PartitionEvaluator API in AggregateInPandasExec, WindowInPandasExec,EvalPythonExec,AttachDistributedSequenceExec Key: SPARK-44362 URL: https://issues.apache.org/jira/browse/SPARK-44362 Project: Spark Issue Type: Sub-task Components: SQL Affects Versions: 3.5.0 Reporter: Vinod KC Use PartitionEvaluator API in AggregateInPandasExec WindowInPandasExec EvalPythonExec AttachDistributedSequenceExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-44361) Use PartitionEvaluator API in BatchEvalPythonUDTFExec, FlatMapGroupsInPandasExec, MapInBatchExec, FlatMapCoGroupsInPandasExec
Vinod KC created SPARK-44361: Summary: Use PartitionEvaluator API in BatchEvalPythonUDTFExec, FlatMapGroupsInPandasExec, MapInBatchExec, FlatMapCoGroupsInPandasExec Key: SPARK-44361 URL: https://issues.apache.org/jira/browse/SPARK-44361 Project: Spark Issue Type: Sub-task Components: SQL Affects Versions: 3.5.0 Reporter: Vinod KC Use PartitionEvaluator API in BatchEvalPythonUDTFExec, FlatMapGroupsInPandasExec, MapInBatchExec, FlatMapCoGroupsInPandasExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-44359) Define the computing logic through PartitionEvaluator API and use it in SQL aggregate operators
[ https://issues.apache.org/jira/browse/SPARK-44359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741736#comment-17741736 ] Vinod KC commented on SPARK-44359: -- Im working on this > Define the computing logic through PartitionEvaluator API and use it in SQL > aggregate operators > --- > > Key: SPARK-44359 > URL: https://issues.apache.org/jira/browse/SPARK-44359 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Define the computing logic through PartitionEvaluator API and use it in SQL > aggregate operators > `MergingSessionsExec` > `SortAggregateExec` > `UpdatingSessionsExec` > `HashAggregateExec` > `ObjectHashAggregateExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-44359) Define the computing logic through PartitionEvaluator API and use it in SQL aggregate operators
Vinod KC created SPARK-44359: Summary: Define the computing logic through PartitionEvaluator API and use it in SQL aggregate operators Key: SPARK-44359 URL: https://issues.apache.org/jira/browse/SPARK-44359 Project: Spark Issue Type: Sub-task Components: SQL Affects Versions: 3.5.0 Reporter: Vinod KC Define the computing logic through PartitionEvaluator API and use it in SQL aggregate operators `MergingSessionsExec` `SortAggregateExec` `UpdatingSessionsExec` `HashAggregateExec` `ObjectHashAggregateExec` -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-44330) Define the computing logic through PartitionEvaluator API and use it in BroadcastNestedLoopJoinExec & BroadcastHashJoinExec
[ https://issues.apache.org/jira/browse/SPARK-44330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741734#comment-17741734 ] Vinod KC commented on SPARK-44330: -- PR raised : https://github.com/apache/spark/pull/41888 > Define the computing logic through PartitionEvaluator API and use it in > BroadcastNestedLoopJoinExec & BroadcastHashJoinExec > --- > > Key: SPARK-44330 > URL: https://issues.apache.org/jira/browse/SPARK-44330 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Define the computing logic through PartitionEvaluator API and use it in > BroadcastNestedLoopJoinExec & BroadcastHashJoinExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-44330) Define the computing logic through PartitionEvaluator API and use it in BroadcastNestedLoopJoinExec & BroadcastHashJoinExec
Vinod KC created SPARK-44330: Summary: Define the computing logic through PartitionEvaluator API and use it in BroadcastNestedLoopJoinExec & BroadcastHashJoinExec Key: SPARK-44330 URL: https://issues.apache.org/jira/browse/SPARK-44330 Project: Spark Issue Type: Sub-task Components: SQL Affects Versions: 3.5.0 Reporter: Vinod KC Define the computing logic through PartitionEvaluator API and use it in BroadcastNestedLoopJoinExec & BroadcastHashJoinExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-44325) Define the computing logic through PartitionEvaluator API and use it in SortMergeJoinExec
Vinod KC created SPARK-44325: Summary: Define the computing logic through PartitionEvaluator API and use it in SortMergeJoinExec Key: SPARK-44325 URL: https://issues.apache.org/jira/browse/SPARK-44325 Project: Spark Issue Type: Sub-task Components: SQL Affects Versions: 3.5.0 Reporter: Vinod KC Define the computing logic through PartitionEvaluator API and use it in SortMergeJoinExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-44317) Define the computing logic through PartitionEvaluator API and use it in ShuffledHashJoinExec
Vinod KC created SPARK-44317: Summary: Define the computing logic through PartitionEvaluator API and use it in ShuffledHashJoinExec Key: SPARK-44317 URL: https://issues.apache.org/jira/browse/SPARK-44317 Project: Spark Issue Type: Sub-task Components: SQL Affects Versions: 3.5.0 Reporter: Vinod KC Define the computing logic through PartitionEvaluator API and use it in ShuffledHashJoinExec -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44286) Define the computing logic through PartitionEvaluator API and use it in SQL operators
[ https://issues.apache.org/jira/browse/SPARK-44286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44286: - Description: Define the computing logic through PartitionEvaluator API and use it in SQL operators. This will avoid lambda-based distributed execution Ref : SPARK-43061. Note: this is an umbrella Jira to apply PartitionEvaluator based approach in SQL operators was: Define the computing logic through PartitionEvaluator API and use it in SQL operators. This will avoid lambda-based distributed execution Ref : SPARK-43061. Note: this is an umbrella Jira to apply PartitionEvaluator base approach in SQL operators > Define the computing logic through PartitionEvaluator API and use it in SQL > operators > - > > Key: SPARK-44286 > URL: https://issues.apache.org/jira/browse/SPARK-44286 > Project: Spark > Issue Type: Umbrella > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Define the computing logic through PartitionEvaluator API and use it in SQL > operators. This will avoid lambda-based distributed execution > Ref : SPARK-43061. > > Note: this is an umbrella Jira to apply PartitionEvaluator based approach in > SQL operators -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-44287) Define the computing logic through PartitionEvaluator API and use it in RowToColumnarExec & ColumnarToRowExec SQL operators.
[ https://issues.apache.org/jira/browse/SPARK-44287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17739755#comment-17739755 ] Vinod KC edited comment on SPARK-44287 at 7/4/23 12:55 AM: --- PR raised: [https://github.com/apache/spark/pull/41839/files] was (Author: vinodkc): PR raised : https://github.com/apache/spark/pull/41839/files > Define the computing logic through PartitionEvaluator API and use it in > RowToColumnarExec & ColumnarToRowExec SQL operators. > - > > Key: SPARK-44287 > URL: https://issues.apache.org/jira/browse/SPARK-44287 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > > Define the computing logic through PartitionEvaluator API and use it in > RowToColumnarExec & ColumnarToRowExec SQL operators. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-44287) Define the computing logic through PartitionEvaluator API and use it in RowToColumnarExec & ColumnarToRowExec SQL operators.
[ https://issues.apache.org/jira/browse/SPARK-44287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17739755#comment-17739755 ] Vinod KC commented on SPARK-44287: -- PR raised : https://github.com/apache/spark/pull/41839/files > Define the computing logic through PartitionEvaluator API and use it in > RowToColumnarExec & ColumnarToRowExec SQL operators. > - > > Key: SPARK-44287 > URL: https://issues.apache.org/jira/browse/SPARK-44287 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > > Define the computing logic through PartitionEvaluator API and use it in > RowToColumnarExec & ColumnarToRowExec SQL operators. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44287) Define the computing logic through PartitionEvaluator API and use it in RowToColumnarExec & ColumnarToRowExec SQL operators.
[ https://issues.apache.org/jira/browse/SPARK-44287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44287: - Summary: Define the computing logic through PartitionEvaluator API and use it in RowToColumnarExec & ColumnarToRowExec SQL operators. (was: Define the computing logic though PartitionEvaluator API and use it in RowToColumnarExec & ColumnarToRowExec SQL operators.) > Define the computing logic through PartitionEvaluator API and use it in > RowToColumnarExec & ColumnarToRowExec SQL operators. > - > > Key: SPARK-44287 > URL: https://issues.apache.org/jira/browse/SPARK-44287 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > > Define the computing logic through PartitionEvaluator API and use it in > RowToColumnarExec & ColumnarToRowExec SQL operators. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44286) Define the computing logic through PartitionEvaluator API and use it in SQL operators
[ https://issues.apache.org/jira/browse/SPARK-44286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44286: - Issue Type: Umbrella (was: Improvement) > Define the computing logic through PartitionEvaluator API and use it in SQL > operators > - > > Key: SPARK-44286 > URL: https://issues.apache.org/jira/browse/SPARK-44286 > Project: Spark > Issue Type: Umbrella > Components: SQL >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Major > > Define the computing logic through PartitionEvaluator API and use it in SQL > operators. This will avoid lambda-based distributed execution > Ref : SPARK-43061. > > Note: this is an umbrella Jira to apply PartitionEvaluator base approach in > SQL operators -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-44287) Define the computing logic though PartitionEvaluator API and use it in RowToColumnarExec & ColumnarToRowExec SQL operators.
Vinod KC created SPARK-44287: Summary: Define the computing logic though PartitionEvaluator API and use it in RowToColumnarExec & ColumnarToRowExec SQL operators. Key: SPARK-44287 URL: https://issues.apache.org/jira/browse/SPARK-44287 Project: Spark Issue Type: Sub-task Components: SQL Affects Versions: 3.5.0 Reporter: Vinod KC Define the computing logic through PartitionEvaluator API and use it in RowToColumnarExec & ColumnarToRowExec SQL operators. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-44286) Define the computing logic through PartitionEvaluator API and use it in SQL operators
Vinod KC created SPARK-44286: Summary: Define the computing logic through PartitionEvaluator API and use it in SQL operators Key: SPARK-44286 URL: https://issues.apache.org/jira/browse/SPARK-44286 Project: Spark Issue Type: Improvement Components: SQL Affects Versions: 3.5.0 Reporter: Vinod KC Define the computing logic through PartitionEvaluator API and use it in SQL operators. This will avoid lambda-based distributed execution Ref : SPARK-43061. Note: this is an umbrella Jira to apply PartitionEvaluator base approach in SQL operators -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44198) Support propagation of the log level to the executors
[ https://issues.apache.org/jira/browse/SPARK-44198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44198: - Summary: Support propagation of the log level to the executors (was: Inconsistent Log Level Setting between Spark Driver and Executors) > Support propagation of the log level to the executors > - > > Key: SPARK-44198 > URL: https://issues.apache.org/jira/browse/SPARK-44198 > Project: Spark > Issue Type: Improvement > Components: Spark Core >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Minor > > Currently, the *sc.setLogLevel()* method only sets the log level on the Spark > driver, failing to reflect the desired log level on the executors. This > inconsistency can lead to difficulties in debugging and monitoring Spark > applications, as log messages from the executors may not align with the > expected log level set on the user code. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-44198) Inconsistent Log Level Setting between Spark Driver and Executors
[ https://issues.apache.org/jira/browse/SPARK-44198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17737308#comment-17737308 ] Vinod KC commented on SPARK-44198: -- Raised PR : https://github.com/apache/spark/pull/41746 > Inconsistent Log Level Setting between Spark Driver and Executors > - > > Key: SPARK-44198 > URL: https://issues.apache.org/jira/browse/SPARK-44198 > Project: Spark > Issue Type: Improvement > Components: Spark Core >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Minor > > Currently, the *sc.setLogLevel()* method only sets the log level on the Spark > driver, failing to reflect the desired log level on the executors. This > inconsistency can lead to difficulties in debugging and monitoring Spark > applications, as log messages from the executors may not align with the > expected log level set on the user code. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-44198) Inconsistent Log Level Setting between Spark Driver and Executors
[ https://issues.apache.org/jira/browse/SPARK-44198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-44198: - Description: Currently, the *sc.setLogLevel()* method only sets the log level on the Spark driver, failing to reflect the desired log level on the executors. This inconsistency can lead to difficulties in debugging and monitoring Spark applications, as log messages from the executors may not align with the expected log level set on the user code. (was: Currently, the *sc.logLevel()* method only sets the log level on the Spark driver, failing to reflect the desired log level on the executors. This inconsistency can lead to difficulties in debugging and monitoring Spark applications, as log messages from the executors may not align with the expected log level set on the user code.) > Inconsistent Log Level Setting between Spark Driver and Executors > - > > Key: SPARK-44198 > URL: https://issues.apache.org/jira/browse/SPARK-44198 > Project: Spark > Issue Type: Improvement > Components: Spark Core >Affects Versions: 3.5.0 >Reporter: Vinod KC >Priority: Minor > > Currently, the *sc.setLogLevel()* method only sets the log level on the Spark > driver, failing to reflect the desired log level on the executors. This > inconsistency can lead to difficulties in debugging and monitoring Spark > applications, as log messages from the executors may not align with the > expected log level set on the user code. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-44198) Inconsistent Log Level Setting between Spark Driver and Executors
Vinod KC created SPARK-44198: Summary: Inconsistent Log Level Setting between Spark Driver and Executors Key: SPARK-44198 URL: https://issues.apache.org/jira/browse/SPARK-44198 Project: Spark Issue Type: Improvement Components: Spark Core Affects Versions: 3.5.0 Reporter: Vinod KC Currently, the *sc.logLevel()* method only sets the log level on the Spark driver, failing to reflect the desired log level on the executors. This inconsistency can lead to difficulties in debugging and monitoring Spark applications, as log messages from the executors may not align with the expected log level set on the user code. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-43470) Add operating system ,Java, Python version information to application log
[ https://issues.apache.org/jira/browse/SPARK-43470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17721885#comment-17721885 ] Vinod KC commented on SPARK-43470: -- https://github.com/apache/spark/pull/41144 > Add operating system ,Java, Python version information to application log > - > > Key: SPARK-43470 > URL: https://issues.apache.org/jira/browse/SPARK-43470 > Project: Spark > Issue Type: Improvement > Components: Spark Core >Affects Versions: 3.4.0 >Reporter: Vinod KC >Priority: Minor > > Include the operating system and Java version, python version information in > the Application log. This will provide useful context and aid in > troubleshooting and debugging any issues that may arise, particularly when > Spark runs across heterogeneous environments (systems with varying operating > systems and Java versions). -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-43470) Add operating system ,Java, Python version information to application log
Vinod KC created SPARK-43470: Summary: Add operating system ,Java, Python version information to application log Key: SPARK-43470 URL: https://issues.apache.org/jira/browse/SPARK-43470 Project: Spark Issue Type: Improvement Components: Spark Core Affects Versions: 3.4.0 Reporter: Vinod KC Include the operating system and Java version, python version information in the Application log. This will provide useful context and aid in troubleshooting and debugging any issues that may arise, particularly when Spark runs across heterogeneous environments (systems with varying operating systems and Java versions). -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-42191) Support udf luhn_check
[ https://issues.apache.org/jira/browse/SPARK-42191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-42191: - Description: Support UDF to check if a given number string is a valid Luhn number. It shall return true if the number string is a valid Luhn number, and false otherwise. This checksum function is widely applied to credit card numbers and government identification numbers to distinguish valid numbers from mistyped, incorrect numbers Ref : [trino|https://trino.io/docs/current/functions/string.html] [Algorithm|https://en.wikipedia.org/wiki/Luhn_algorithm] was: Support UDF to check if a given number string is a valid Luhn number. It shall return true if the number string is a valid Luhn number, and false otherwise. This checksum function is widely applied to credit card numbers and government identification numbers to distinguish valid numbers from mistyped, incorrect numbers Ref : [trino|https://trino.io/docs/current/functions/string.html] > Support udf luhn_check > --- > > Key: SPARK-42191 > URL: https://issues.apache.org/jira/browse/SPARK-42191 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.4.0 >Reporter: Vinod KC >Priority: Minor > > Support UDF to check if a given number string is a valid Luhn number. It > shall return true if the number string is a valid Luhn number, and false > otherwise. > This checksum function is widely applied to credit card numbers and > government identification numbers to distinguish valid numbers from mistyped, > incorrect numbers > Ref : [trino|https://trino.io/docs/current/functions/string.html] > [Algorithm|https://en.wikipedia.org/wiki/Luhn_algorithm] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-42191) Support udf luhn_check
Vinod KC created SPARK-42191: Summary: Support udf luhn_check Key: SPARK-42191 URL: https://issues.apache.org/jira/browse/SPARK-42191 Project: Spark Issue Type: Sub-task Components: SQL Affects Versions: 3.4.0 Reporter: Vinod KC Support UDF to check if a given number string is a valid Luhn number. It shall return true if the number string is a valid Luhn number, and false otherwise. This checksum function is widely applied to credit card numbers and government identification numbers to distinguish valid numbers from mistyped, incorrect numbers Ref : [trino|https://trino.io/docs/current/functions/string.html] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-40687) Support data masking built-in Function 'mask'
[
https://issues.apache.org/jira/browse/SPARK-40687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17677988#comment-17677988
]
Vinod KC edited comment on SPARK-40687 at 1/17/23 9:29 PM:
---
Note: In the udf 'mask', using -1 as ignore parameter in String type argument
is not a standard way.
Please refer SPARK-42070 , it changes the default value of the argument of
m{*}ask{*} udf from -1 to NULL
was (Author: vinodkc):
Note: Please refer
[SPARK-42070|https://issues.apache.org/jira/browse/SPARK-42070] , it changes
the default value of argument of Mask udf from -1 to NULL
> Support data masking built-in Function 'mask'
> --
>
> Key: SPARK-40687
> URL: https://issues.apache.org/jira/browse/SPARK-40687
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: Vinod KC
>Assignee: Vinod KC
>Priority: Minor
> Fix For: 3.4.0
>
>
> Support data masking built-in Function *mask*
> Return a masked version of str. By default, upper case letters should be
> converted to "X", lower case letters should be converted to "x" and numbers
> should be converted to "n". For example mask("abcd-EFGH-8765-4321") results
> in ---. Should be able override the characters used in the
> mask by supplying additional arguments: the second argument controls the mask
> character for upper case letters, the third argument for lower case letters
> and the fourth argument for numbers. For example, mask("abcd-EFGH-8765-4321",
> "U", "l", "#") should result in ---
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-40687) Support data masking built-in Function 'mask'
[
https://issues.apache.org/jira/browse/SPARK-40687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17677988#comment-17677988
]
Vinod KC commented on SPARK-40687:
--
Note: Please refer
[SPARK-42070|https://issues.apache.org/jira/browse/SPARK-42070] , it changes
the default value of argument of Mask udf from -1 to NULL
> Support data masking built-in Function 'mask'
> --
>
> Key: SPARK-40687
> URL: https://issues.apache.org/jira/browse/SPARK-40687
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.4.0
>Reporter: Vinod KC
>Assignee: Vinod KC
>Priority: Minor
> Fix For: 3.4.0
>
>
> Support data masking built-in Function *mask*
> Return a masked version of str. By default, upper case letters should be
> converted to "X", lower case letters should be converted to "x" and numbers
> should be converted to "n". For example mask("abcd-EFGH-8765-4321") results
> in ---. Should be able override the characters used in the
> mask by supplying additional arguments: the second argument controls the mask
> character for upper case letters, the third argument for lower case letters
> and the fourth argument for numbers. For example, mask("abcd-EFGH-8765-4321",
> "U", "l", "#") should result in ---
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-42070) Change the default value of argument of Mask udf from -1 to NULL
[ https://issues.apache.org/jira/browse/SPARK-42070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-42070: - Description: In the udf 'mask', using -1 as ignore parameter in String type argument is not a standard way, hence, it is better to change the value of ignore argument from -1 to NULL. Note: SPARK-40687 , has recently implemented udf *mask* , which uses -1 as the default argument to ignore the masking option, As no Spark version release has occurred since then, this new change will not cause backward compatibility issues was: In the udf 'mask', using -1 as ignore parameter in String type argument is not a standard way, hence it is better to change the value of ignore argument from -1 to NULL. > Change the default value of argument of Mask udf from -1 to NULL > > > Key: SPARK-42070 > URL: https://issues.apache.org/jira/browse/SPARK-42070 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.4.0 >Reporter: Vinod KC >Priority: Minor > > In the udf 'mask', using -1 as ignore parameter in String type argument is > not a standard way, hence, it is better to change the value of ignore > argument from -1 to NULL. > Note: SPARK-40687 , has recently implemented udf *mask* , which uses -1 as > the default argument to ignore the masking option, As no Spark version > release has occurred since then, this new change will not cause backward > compatibility issues -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-42070) Change the default value of argument of Mask udf from -1 to NULL
Vinod KC created SPARK-42070: Summary: Change the default value of argument of Mask udf from -1 to NULL Key: SPARK-42070 URL: https://issues.apache.org/jira/browse/SPARK-42070 Project: Spark Issue Type: Sub-task Components: SQL Affects Versions: 3.4.0 Reporter: Vinod KC In the udf 'mask', using -1 as ignore parameter in String type argument is not a standard way, hence it is better to change the value of ignore argument from -1 to NULL. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-41079) Support Bit manipulation built-in functions to perform bitwise operations
[ https://issues.apache.org/jira/browse/SPARK-41079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-41079: - Summary: Support Bit manipulation built-in functions to perform bitwise operations (was: Support Bit manipulation functions to perform bitwise operations) > Support Bit manipulation built-in functions to perform bitwise operations > - > > Key: SPARK-41079 > URL: https://issues.apache.org/jira/browse/SPARK-41079 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.1.4 >Reporter: Vinod KC >Priority: Minor > > Support Bit manipulation functions to perform bitwise operations > COUNTSET > ROTATELEFT > ROTATERIGHT > SETBIT > SHIFTLEFT > SHIFTRIGHT > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-41085) Support Bit manipulation function COUNTSET
[ https://issues.apache.org/jira/browse/SPARK-41085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17631356#comment-17631356 ] Vinod KC commented on SPARK-41085: -- I'm working on this > Support Bit manipulation function COUNTSET > -- > > Key: SPARK-41085 > URL: https://issues.apache.org/jira/browse/SPARK-41085 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.1.4 >Reporter: Vinod KC >Priority: Minor > > Support Bit manipulation function COUNTSET > The function shall return the number of 1 bits in the specified integer > value. If the optional second argument is set to zero, it shall return the > number of 0 bits instead. > COUNTSET(integer_type a [, INT zero_or_one]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-41084) Support Bit manipulation function SHIFTRIGHT
[ https://issues.apache.org/jira/browse/SPARK-41084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17631355#comment-17631355 ] Vinod KC commented on SPARK-41084: -- I'm working on this > Support Bit manipulation function SHIFTRIGHT > > > Key: SPARK-41084 > URL: https://issues.apache.org/jira/browse/SPARK-41084 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.1.4 >Reporter: Vinod KC >Priority: Minor > > Support Bit manipulation function SHIFTRIGHT > The function shall shift an integer value right by a specified number of > bits. As the least significant bit will be taken out of the original value, > it shall be discarded and the most significant bit becomes 0. > *SHIFTRIGHT(integer_type a, INT positions)* -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-41083) Support Bit manipulation function SHIFTLEFT
[ https://issues.apache.org/jira/browse/SPARK-41083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17631353#comment-17631353 ] Vinod KC commented on SPARK-41083: -- I'm working on this > Support Bit manipulation function SHIFTLEFT > --- > > Key: SPARK-41083 > URL: https://issues.apache.org/jira/browse/SPARK-41083 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.1.4 >Reporter: Vinod KC >Priority: Major > > The function shall shift an integer value left by a specified number of bits. > As the most significant bit will be taken out of the original value, it shall > discard and the least significant bit becomes 0 > SHIFTLEFT(integer_type a, INT positions) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-41083) Support Bit manipulation function SHIFTLEFT
[ https://issues.apache.org/jira/browse/SPARK-41083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-41083: - Description: The function shall shift an integer value left by a specified number of bits. As the most significant bit will be taken out of the original value, it shall discard and the least significant bit becomes 0 SHIFTLEFT(integer_type a, INT positions) was: The function shall shift an integer value left by a specified number of bits. As the most significant bit will be taken out of the original value, it should discard and the least significant bit becomes 0 SHIFTLEFT(integer_type a, INT positions) > Support Bit manipulation function SHIFTLEFT > --- > > Key: SPARK-41083 > URL: https://issues.apache.org/jira/browse/SPARK-41083 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.1.4 >Reporter: Vinod KC >Priority: Major > > The function shall shift an integer value left by a specified number of bits. > As the most significant bit will be taken out of the original value, it shall > discard and the least significant bit becomes 0 > SHIFTLEFT(integer_type a, INT positions) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-41082) Support Bit manipulation function ROTATERIGHT
[ https://issues.apache.org/jira/browse/SPARK-41082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17631352#comment-17631352 ] Vinod KC commented on SPARK-41082: -- I'm working on this > Support Bit manipulation function ROTATERIGHT > - > > Key: SPARK-41082 > URL: https://issues.apache.org/jira/browse/SPARK-41082 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.1.4 >Reporter: Vinod KC >Priority: Major > > Support Bit manipulation function ROTATERIGHT > The function shall rotate an integer value right by a specified number of > bits. As the least significant bit will be taken out of the original value, > if it is a 1 bit, it shall be "rotated" back to the most significant bit. > Therefore, the final value will have the same number of 1 bits as the > original value, just in different positions > ROTATERIGHT(integer_type a, INT positions) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-41082) Support Bit manipulation function ROTATERIGHT
[ https://issues.apache.org/jira/browse/SPARK-41082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-41082: - Description: Support Bit manipulation function ROTATERIGHT The function shall rotate an integer value right by a specified number of bits. As the least significant bit will be taken out of the original value, if it is a 1 bit, it shall be "rotated" back to the most significant bit. Therefore, the final value will have the same number of 1 bits as the original value, just in different positions ROTATERIGHT(integer_type a, INT positions) was: Support Bit manipulation function ROTATERIGHT The function shall rotate an integer value right by a specified number of bits. As the least significant bit is taken out of the original value, if it is a 1 bit, it is "rotated" back to the most significant bit. Therefore, the final value has the same number of 1 bits as the original value, just in different positions ROTATERIGHT(integer_type a, INT positions) > Support Bit manipulation function ROTATERIGHT > - > > Key: SPARK-41082 > URL: https://issues.apache.org/jira/browse/SPARK-41082 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.1.4 >Reporter: Vinod KC >Priority: Major > > Support Bit manipulation function ROTATERIGHT > The function shall rotate an integer value right by a specified number of > bits. As the least significant bit will be taken out of the original value, > if it is a 1 bit, it shall be "rotated" back to the most significant bit. > Therefore, the final value will have the same number of 1 bits as the > original value, just in different positions > ROTATERIGHT(integer_type a, INT positions) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-41081) Support Bit manipulation function ROTATELEFT
[ https://issues.apache.org/jira/browse/SPARK-41081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-41081: - Description: Support Bit manipulation function ROTATELEFT The function shall rotate an integer value left by a specified number of bits. As the most significant bit will be taken out of the original value, if it is a 1 bit, it shall be "rotated" back to the least significant bit. *ROTATELEFT(integer_type a, INT positions)* was: Support Bit manipulation function ROTATELEFT The function shall rotate an integer value left by a specified number of bits. As the most significant bit is taken out of the original value, if it is a 1 bit, it is "rotated" back to the least significant bit. *ROTATELEFT(integer_type a, INT positions)* > Support Bit manipulation function ROTATELEFT > > > Key: SPARK-41081 > URL: https://issues.apache.org/jira/browse/SPARK-41081 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.1.4 >Reporter: Vinod KC >Priority: Major > > Support Bit manipulation function ROTATELEFT > The function shall rotate an integer value left by a specified number of > bits. As the most significant bit will be taken out of the original value, if > it is a 1 bit, it shall be "rotated" back to the least significant bit. > *ROTATELEFT(integer_type a, INT positions)* -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-41081) Support Bit manipulation function ROTATELEFT
[ https://issues.apache.org/jira/browse/SPARK-41081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17631351#comment-17631351 ] Vinod KC commented on SPARK-41081: -- I'm working on this > Support Bit manipulation function ROTATELEFT > > > Key: SPARK-41081 > URL: https://issues.apache.org/jira/browse/SPARK-41081 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.1.4 >Reporter: Vinod KC >Priority: Major > > Support Bit manipulation function ROTATELEFT > The function shall rotate an integer value left by a specified number of > bits. As the most significant bit will be taken out of the original value, if > it is a 1 bit, it shall be "rotated" back to the least significant bit. > *ROTATELEFT(integer_type a, INT positions)* -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-41080) Support Bit manipulation function SETBIT
[ https://issues.apache.org/jira/browse/SPARK-41080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17631349#comment-17631349 ] Vinod KC commented on SPARK-41080: -- Im working on this > Support Bit manipulation function SETBIT > > > Key: SPARK-41080 > URL: https://issues.apache.org/jira/browse/SPARK-41080 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.1.4 >Reporter: Vinod KC >Priority: Minor > > Support function to change a bit at a specified position. It shall change a > bit at a specified position to a 1, If the optional third argument is set to > zero, the specified bit shall be set to 0 instead. > SETBIT(integer_type a, INT position [, INT zero_or_one]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-41080) Support Bit manipulation function SETBIT
[ https://issues.apache.org/jira/browse/SPARK-41080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-41080: - Description: Support function to change a bit at a specified position. It shall change a bit at a specified position to a 1, If the optional third argument is set to zero, the specified bit shall be set to 0 instead. SETBIT(integer_type a, INT position [, INT zero_or_one]) was: Support function to change a bit at a specified position SETBIT(integer_type a, INT position [, INT zero_or_one]) > Support Bit manipulation function SETBIT > > > Key: SPARK-41080 > URL: https://issues.apache.org/jira/browse/SPARK-41080 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.1.4 >Reporter: Vinod KC >Priority: Minor > > Support function to change a bit at a specified position. It shall change a > bit at a specified position to a 1, If the optional third argument is set to > zero, the specified bit shall be set to 0 instead. > SETBIT(integer_type a, INT position [, INT zero_or_one]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-41085) Support Bit manipulation function COUNTSET
Vinod KC created SPARK-41085: Summary: Support Bit manipulation function COUNTSET Key: SPARK-41085 URL: https://issues.apache.org/jira/browse/SPARK-41085 Project: Spark Issue Type: Sub-task Components: SQL Affects Versions: 3.1.4 Reporter: Vinod KC Support Bit manipulation function COUNTSET The function shall return the number of 1 bits in the specified integer value. If the optional second argument is set to zero, it shall return the number of 0 bits instead. COUNTSET(integer_type a [, INT zero_or_one]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-41079) Support Bit manipulation functions to perform bitwise operations
[ https://issues.apache.org/jira/browse/SPARK-41079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-41079: - Summary: Support Bit manipulation functions to perform bitwise operations (was: Support Bit manipulation functions perform bitwise operations) > Support Bit manipulation functions to perform bitwise operations > > > Key: SPARK-41079 > URL: https://issues.apache.org/jira/browse/SPARK-41079 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.1.4 >Reporter: Vinod KC >Priority: Minor > > Support Bit manipulation functions to perform bitwise operations > COUNTSET > ROTATELEFT > ROTATERIGHT > SETBIT > SHIFTLEFT > SHIFTRIGHT > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
