[jira] [Updated] (SPARK-44458) ThreadLocal not being copied to child thread when child thread is reused from pool

2023-07-17 Thread Kapil Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kapil Singh updated SPARK-44458: Affects Version/s: 3.1.0 (was: 3.4.1) > ThreadLocal not being copied

[jira] [Updated] (SPARK-44458) ThreadLocal not being copied to child thread when child thread is reused from pool

2023-07-17 Thread Kapil Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kapil Singh updated SPARK-44458: Attachment: (was: image-2023-07-17-16-57-09-669.png) > ThreadLocal not being copied to child

[jira] [Updated] (SPARK-44458) ThreadLocal not being copied to child thread when child thread is reused from pool

2023-07-17 Thread Kapil Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kapil Singh updated SPARK-44458: Attachment: (was: image-2023-07-17-16-56-59-388.png) > ThreadLocal not being copied to child

[jira] [Updated] (SPARK-44458) ThreadLocal not being copied to child thread when child thread is reused from pool

2023-07-17 Thread Kapil Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kapil Singh updated SPARK-44458: Description: Currently setting the Spark configuration using the statement

[jira] [Updated] (SPARK-44458) ThreadLocal not being copied to child thread when child thread is reused from pool

2023-07-17 Thread Kapil Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kapil Singh updated SPARK-44458: Description: Currently setting the Spark configuration using the statement

[jira] [Updated] (SPARK-44458) ThreadLocal not being copied to child thread when child thread is reused from pool

2023-07-17 Thread Kapil Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kapil Singh updated SPARK-44458: Attachment: image-2023-07-17-16-56-59-388.png > ThreadLocal not being copied to child thread when

[jira] [Updated] (SPARK-44458) ThreadLocal not being copied to child thread when child thread is reused from pool

2023-07-17 Thread Kapil Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kapil Singh updated SPARK-44458: Attachment: image-2023-07-17-16-57-09-669.png > ThreadLocal not being copied to child thread when

[jira] [Created] (SPARK-44458) ThreadLocal not being copied to child thread when child thread is reused from pool

2023-07-17 Thread Kapil Singh (Jira)
Kapil Singh created SPARK-44458: --- Summary: ThreadLocal not being copied to child thread when child thread is reused from pool Key: SPARK-44458 URL: https://issues.apache.org/jira/browse/SPARK-44458

[jira] [Updated] (SPARK-44077) Session Configs were not getting honored in RDDs

2023-06-15 Thread Kapil Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kapil Singh updated SPARK-44077: Description: When calling SQLConf.get on executors, the configs are read from the local

[jira] [Created] (SPARK-44077) Session Configs were not getting honored in RDDs

2023-06-15 Thread Kapil Singh (Jira)
Kapil Singh created SPARK-44077: --- Summary: Session Configs were not getting honored in RDDs Key: SPARK-44077 URL: https://issues.apache.org/jira/browse/SPARK-44077 Project: Spark Issue Type:

[jira] [Created] (SPARK-42660) Infer filters for Join produced by IN and EXISTS clause (RewritePredicateSubquery rule)

2023-03-02 Thread Kapil Singh (Jira)
Kapil Singh created SPARK-42660: --- Summary: Infer filters for Join produced by IN and EXISTS clause (RewritePredicateSubquery rule) Key: SPARK-42660 URL: https://issues.apache.org/jira/browse/SPARK-42660

[jira] [Created] (SPARK-42598) Refactor TPCH schema to separate file similar to TPCDS for code reuse

2023-02-27 Thread Kapil Singh (Jira)
Kapil Singh created SPARK-42598: --- Summary: Refactor TPCH schema to separate file similar to TPCDS for code reuse Key: SPARK-42598 URL: https://issues.apache.org/jira/browse/SPARK-42598 Project: Spark

[jira] [Created] (SPARK-40558) Add Reusable Exchange in Bloom creation side plan

2022-09-26 Thread Kapil Singh (Jira)
Kapil Singh created SPARK-40558: --- Summary: Add Reusable Exchange in Bloom creation side plan Key: SPARK-40558 URL: https://issues.apache.org/jira/browse/SPARK-40558 Project: Spark Issue Type:

[jira] [Created] (SPARK-40231) Add 1TB TPCDS Plan stability tests

2022-08-26 Thread Kapil Singh (Jira)
Kapil Singh created SPARK-40231: --- Summary: Add 1TB TPCDS Plan stability tests Key: SPARK-40231 URL: https://issues.apache.org/jira/browse/SPARK-40231 Project: Spark Issue Type: Task

[jira] [Updated] (SPARK-40124) Update TPCDS v1.4 q32 for Plan Stability tests

2022-08-17 Thread Kapil Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kapil Singh updated SPARK-40124: Summary: Update TPCDS v1.4 q32 for Plan Stability tests (was: Update TPCDS v1.4 query32) >

[jira] [Created] (SPARK-40124) Update TPCDS v1.4 query32

2022-08-17 Thread Kapil Singh (Jira)
Kapil Singh created SPARK-40124: --- Summary: Update TPCDS v1.4 query32 Key: SPARK-40124 URL: https://issues.apache.org/jira/browse/SPARK-40124 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-39690) Reuse exchange across subqueries is broken with AQE if subquery side exchange materialized first

2022-07-06 Thread Kapil Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kapil Singh updated SPARK-39690: Description: When trying to reuse Exchange of a subquery in main plan, if the Exchange inside

[jira] [Created] (SPARK-39690) Reuse exchange across subqueries is broken with AQE if subquery side exchange materialized first

2022-07-05 Thread Kapil Singh (Jira)
Kapil Singh created SPARK-39690: --- Summary: Reuse exchange across subqueries is broken with AQE if subquery side exchange materialized first Key: SPARK-39690 URL: https://issues.apache.org/jira/browse/SPARK-39690

[jira] [Commented] (SPARK-37995) TPCDS 1TB q72 fails when spark.sql.optimizer.dynamicPartitionPruning.reuseBroadcastOnly is false

2022-01-31 Thread Kapil Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17485039#comment-17485039 ] Kapil Singh commented on SPARK-37995: - Hey folks, any input on this issue? [~hyukjin.kwon]

[jira] [Comment Edited] (SPARK-37995) TPCDS 1TB q72 fails when spark.sql.optimizer.dynamicPartitionPruning.reuseBroadcastOnly is false

2022-01-24 Thread Kapil Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17481561#comment-17481561 ] Kapil Singh edited comment on SPARK-37995 at 1/25/22, 6:12 AM: --- Seems it

[jira] [Commented] (SPARK-37995) TPCDS 1TB q72 fails when spark.sql.optimizer.dynamicPartitionPruning.reuseBroadcastOnly is false

2022-01-24 Thread Kapil Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17481561#comment-17481561 ] Kapil Singh commented on SPARK-37995: - Seems it is related to this part of

[jira] [Updated] (SPARK-37995) TPCDS 1TB q72 fails when spark.sql.optimizer.dynamicPartitionPruning.reuseBroadcastOnly is false

2022-01-24 Thread Kapil Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kapil Singh updated SPARK-37995: Attachment: full-stacktrace.txt > TPCDS 1TB q72 fails when >

[jira] [Created] (SPARK-37995) TPCDS 1TB q72 fails when spark.sql.optimizer.dynamicPartitionPruning.reuseBroadcastOnly is false

2022-01-24 Thread Kapil Singh (Jira)
Kapil Singh created SPARK-37995: --- Summary: TPCDS 1TB q72 fails when spark.sql.optimizer.dynamicPartitionPruning.reuseBroadcastOnly is false Key: SPARK-37995 URL: https://issues.apache.org/jira/browse/SPARK-37995

[jira] [Commented] (SPARK-16894) take function for returning the first n elements of array column

2016-11-10 Thread Kapil Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15653494#comment-15653494 ] Kapil Singh commented on SPARK-16894: - How is it different from array_contains? Why is array_contains

[jira] [Commented] (SPARK-16892) flatten function to get flat array (or map) column from array of array (or array of map) column

2016-11-07 Thread Kapil Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15646584#comment-15646584 ] Kapil Singh commented on SPARK-16892: - It's not for flattening Rows. It's for flattening columns. The

[jira] [Commented] (SPARK-16894) take function for returning the first n elements of array column

2016-11-07 Thread Kapil Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15646576#comment-15646576 ] Kapil Singh commented on SPARK-16894: - This is not about selecting first n elements/columns from a

[jira] [Commented] (SPARK-16894) take function for returning the first n elements of array column

2016-11-07 Thread Kapil Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15646563#comment-15646563 ] Kapil Singh commented on SPARK-16894: - The use case is similar to scala collection's take method. So,

[jira] [Commented] (SPARK-18091) Deep if expressions cause Generated SpecificUnsafeProjection code to exceed JVM code size limit

2016-10-25 Thread Kapil Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15604587#comment-15604587 ] Kapil Singh commented on SPARK-18091: - I've started working on this > Deep if expressions cause

[jira] [Updated] (SPARK-18091) Deep if expressions cause Generated SpecificUnsafeProjection code to exceed JVM code size limit

2016-10-25 Thread Kapil Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kapil Singh updated SPARK-18091: Description: *Problem Description:* I have an application in which a lot of if-else decisioning is

[jira] [Created] (SPARK-18091) Deep if expressions cause Generated SpecificUnsafeProjection code to exceed JVM code size limit

2016-10-25 Thread Kapil Singh (JIRA)
Kapil Singh created SPARK-18091: --- Summary: Deep if expressions cause Generated SpecificUnsafeProjection code to exceed JVM code size limit Key: SPARK-18091 URL: https://issues.apache.org/jira/browse/SPARK-18091

[jira] [Updated] (SPARK-18090) NegativeArraySize exception while reading parquet when inferred type and provided type for partition column are different

2016-10-25 Thread Kapil Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kapil Singh updated SPARK-18090: Description: *Problem Description:* Reading a small parquet file (single column, single record),

[jira] [Updated] (SPARK-18090) NegativeArraySize exception while reading parquet when inferred type and provided type for partition column are different

2016-10-25 Thread Kapil Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kapil Singh updated SPARK-18090: Description: *Problem Description:* Reading a small parquet file (single column, single record),

[jira] [Created] (SPARK-18090) NegativeArraySize exception while reading parquet when inferred type and provided type for partition column are different

2016-10-25 Thread Kapil Singh (JIRA)
Kapil Singh created SPARK-18090: --- Summary: NegativeArraySize exception while reading parquet when inferred type and provided type for partition column are different Key: SPARK-18090 URL:

[jira] [Commented] (SPARK-16889) Add formatMessage Column expression for formatting strings in java.text.MessageFormat style in Scala API

2016-08-08 Thread Kapil Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15411808#comment-15411808 ] Kapil Singh commented on SPARK-16889: - What if the pattern is user-provided or worse if it's a column

[jira] [Created] (SPARK-16894) take function for returning the first n elements of array column

2016-08-04 Thread Kapil Singh (JIRA)
Kapil Singh created SPARK-16894: --- Summary: take function for returning the first n elements of array column Key: SPARK-16894 URL: https://issues.apache.org/jira/browse/SPARK-16894 Project: Spark

[jira] [Created] (SPARK-16892) flatten function to get flat array (or map) column from array of array (or array of map) column

2016-08-04 Thread Kapil Singh (JIRA)
Kapil Singh created SPARK-16892: --- Summary: flatten function to get flat array (or map) column from array of array (or array of map) column Key: SPARK-16892 URL: https://issues.apache.org/jira/browse/SPARK-16892

[jira] [Created] (SPARK-16891) arrayFilter function for filtering elements of array column based on a predicate

2016-08-04 Thread Kapil Singh (JIRA)
Kapil Singh created SPARK-16891: --- Summary: arrayFilter function for filtering elements of array column based on a predicate Key: SPARK-16891 URL: https://issues.apache.org/jira/browse/SPARK-16891

[jira] [Created] (SPARK-16890) substring returns wrong result for positive position

2016-08-04 Thread Kapil Singh (JIRA)
Kapil Singh created SPARK-16890: --- Summary: substring returns wrong result for positive position Key: SPARK-16890 URL: https://issues.apache.org/jira/browse/SPARK-16890 Project: Spark Issue

[jira] [Created] (SPARK-16889) Add formatMessage Column expression for formatting strings in java.text.MessageFormat style in Scala API

2016-08-04 Thread Kapil Singh (JIRA)
Kapil Singh created SPARK-16889: --- Summary: Add formatMessage Column expression for formatting strings in java.text.MessageFormat style in Scala API Key: SPARK-16889 URL:

[jira] [Commented] (SPARK-16876) Add match Column expression for regular expression matching in Scala API

2016-08-03 Thread Kapil Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15407200#comment-15407200 ] Kapil Singh commented on SPARK-16876: - Yes these are similar, but SPARK-16203 is for PySpark while

[jira] [Commented] (SPARK-16876) Add match Column expression for regular expression matching in Scala API

2016-08-03 Thread Kapil Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405961#comment-15405961 ] Kapil Singh commented on SPARK-16876: - For example, consider a data field in which miscellaneous

[jira] [Created] (SPARK-16876) Add match Column expression for regular expression matching in Scala API

2016-08-03 Thread Kapil Singh (JIRA)
Kapil Singh created SPARK-16876: --- Summary: Add match Column expression for regular expression matching in Scala API Key: SPARK-16876 URL: https://issues.apache.org/jira/browse/SPARK-16876 Project:

[jira] [Commented] (SPARK-16876) Add match Column expression for regular expression matching in Scala API

2016-08-03 Thread Kapil Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405808#comment-15405808 ] Kapil Singh commented on SPARK-16876: - I'm working on this. > Add match Column expression for

[jira] [Commented] (SPARK-12194) Add Sink for reporting Spark Metrics to OpenTSDB

2015-12-07 Thread Kapil Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046331#comment-15046331 ] Kapil Singh commented on SPARK-12194: - I'm working on this. > Add Sink for reporting Spark Metrics

[jira] [Created] (SPARK-12194) Add Sink for reporting Spark Metrics to OpenTSDB

2015-12-07 Thread Kapil Singh (JIRA)
Kapil Singh created SPARK-12194: --- Summary: Add Sink for reporting Spark Metrics to OpenTSDB Key: SPARK-12194 URL: https://issues.apache.org/jira/browse/SPARK-12194 Project: Spark Issue Type:

[jira] [Commented] (SPARK-11447) Null comparison requires type information but type extraction fails for complex types

2015-11-03 Thread Kapil Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14987096#comment-14987096 ] Kapil Singh commented on SPARK-11447: - On second look, I seem to have identified the issue. Take a

[jira] [Commented] (SPARK-11447) Null comparison requires type information but type extraction fails for complex types

2015-11-02 Thread Kapil Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14986552#comment-14986552 ] Kapil Singh commented on SPARK-11447: - Hi Kevin, My bad, should have been more explicit. The code

[jira] [Created] (SPARK-11447) Null comparison requires type information but type extraction fails for complex types

2015-11-01 Thread Kapil Singh (JIRA)
Kapil Singh created SPARK-11447: --- Summary: Null comparison requires type information but type extraction fails for complex types Key: SPARK-11447 URL: https://issues.apache.org/jira/browse/SPARK-11447