[jira] [Created] (SPARK-46055) Refactor Catalog Database APIs implementation

2023-11-22 Thread Yihong He (Jira)
Yihong He created SPARK-46055: - Summary: Refactor Catalog Database APIs implementation Key: SPARK-46055 URL: https://issues.apache.org/jira/browse/SPARK-46055 Project: Spark Issue Type:

[jira] [Created] (SPARK-45899) Set errorClass in errorInfoToThrowable

2023-11-12 Thread Yihong He (Jira)
Yihong He created SPARK-45899: - Summary: Set errorClass in errorInfoToThrowable Key: SPARK-45899 URL: https://issues.apache.org/jira/browse/SPARK-45899 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-45898) Rewrite catalog table APIs to use unresolved logical plan

2023-11-12 Thread Yihong He (Jira)
Yihong He created SPARK-45898: - Summary: Rewrite catalog table APIs to use unresolved logical plan Key: SPARK-45898 URL: https://issues.apache.org/jira/browse/SPARK-45898 Project: Spark Issue

[jira] [Created] (SPARK-45842) Refactor Catalog Function APIs to use analyzer

2023-11-08 Thread Yihong He (Jira)
Yihong He created SPARK-45842: - Summary: Refactor Catalog Function APIs to use analyzer Key: SPARK-45842 URL: https://issues.apache.org/jira/browse/SPARK-45842 Project: Spark Issue Type:

[jira] [Created] (SPARK-45726) Make Dataset.collectResult private

2023-10-30 Thread Yihong He (Jira)
Yihong He created SPARK-45726: - Summary: Make Dataset.collectResult private Key: SPARK-45726 URL: https://issues.apache.org/jira/browse/SPARK-45726 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-45609) Include SqlState in SparkThrowable proto message

2023-10-19 Thread Yihong He (Jira)
Yihong He created SPARK-45609: - Summary: Include SqlState in SparkThrowable proto message Key: SPARK-45609 URL: https://issues.apache.org/jira/browse/SPARK-45609 Project: Spark Issue Type: Test

[jira] [Created] (SPARK-45520) Add property testing for error constructors in scala clients

2023-10-12 Thread Yihong He (Jira)
Yihong He created SPARK-45520: - Summary: Add property testing for error constructors in scala clients Key: SPARK-45520 URL: https://issues.apache.org/jira/browse/SPARK-45520 Project: Spark

[jira] [Updated] (SPARK-45518) Error framework support for Python Spark Connect Client

2023-10-12 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-45518: -- Description: * Define APIs for the error framework, especially where to expose QueryContext *

[jira] [Updated] (SPARK-45517) Expand more exception constructors to support error framework parameters

2023-10-12 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-45517: -- Description: * SparkNumberFormatException * SparkIllegalArgumentException *

[jira] [Created] (SPARK-45518) Error framework support for Python Spark Connect Client

2023-10-12 Thread Yihong He (Jira)
Yihong He created SPARK-45518: - Summary: Error framework support for Python Spark Connect Client Key: SPARK-45518 URL: https://issues.apache.org/jira/browse/SPARK-45518 Project: Spark Issue

[jira] [Created] (SPARK-45517) Expand more exception constructors to support error framework parameters

2023-10-12 Thread Yihong He (Jira)
Yihong He created SPARK-45517: - Summary: Expand more exception constructors to support error framework parameters Key: SPARK-45517 URL: https://issues.apache.org/jira/browse/SPARK-45517 Project: Spark

[jira] [Created] (SPARK-45516) Include QueryContext in SparkThrowable proto message

2023-10-12 Thread Yihong He (Jira)
Yihong He created SPARK-45516: - Summary: Include QueryContext in SparkThrowable proto message Key: SPARK-45516 URL: https://issues.apache.org/jira/browse/SPARK-45516 Project: Spark Issue Type:

[jira] [Created] (SPARK-45360) Initialize spark session builder configuration from SPARK_REMOTE

2023-09-27 Thread Yihong He (Jira)
Yihong He created SPARK-45360: - Summary: Initialize spark session builder configuration from SPARK_REMOTE Key: SPARK-45360 URL: https://issues.apache.org/jira/browse/SPARK-45360 Project: Spark

[jira] [Created] (SPARK-45347) Include SparkThrowable in FetchErrorDetailsResponse

2023-09-26 Thread Yihong He (Jira)
Yihong He created SPARK-45347: - Summary: Include SparkThrowable in FetchErrorDetailsResponse Key: SPARK-45347 URL: https://issues.apache.org/jira/browse/SPARK-45347 Project: Spark Issue Type:

[jira] [Created] (SPARK-45317) Handle null filename in stack traces of exceptions"

2023-09-25 Thread Yihong He (Jira)
Yihong He created SPARK-45317: - Summary: Handle null filename in stack traces of exceptions" Key: SPARK-45317 URL: https://issues.apache.org/jira/browse/SPARK-45317 Project: Spark Issue Type:

[jira] [Updated] (SPARK-45317) Handle null filename in stack traces of exceptions

2023-09-25 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-45317: -- Summary: Handle null filename in stack traces of exceptions (was: Handle null filename in stack

[jira] [Created] (SPARK-45251) Add client_type field for FetchErrorDetails

2023-09-21 Thread Yihong He (Jira)
Yihong He created SPARK-45251: - Summary: Add client_type field for FetchErrorDetails Key: SPARK-45251 URL: https://issues.apache.org/jira/browse/SPARK-45251 Project: Spark Issue Type: New

[jira] [Updated] (SPARK-45207) Implement Error Enrichment for Scala Client

2023-09-20 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-45207: -- Summary: Implement Error Enrichment for Scala Client (was: Implement Complete Error Reconstruction

[jira] [Created] (SPARK-45240) Implement Error Enrichment for Python Client

2023-09-20 Thread Yihong He (Jira)
Yihong He created SPARK-45240: - Summary: Implement Error Enrichment for Python Client Key: SPARK-45240 URL: https://issues.apache.org/jira/browse/SPARK-45240 Project: Spark Issue Type: New

[jira] [Created] (SPARK-45239) Reduce default spark.connect.jvmStacktrace.maxSize

2023-09-20 Thread Yihong He (Jira)
Yihong He created SPARK-45239: - Summary: Reduce default spark.connect.jvmStacktrace.maxSize Key: SPARK-45239 URL: https://issues.apache.org/jira/browse/SPARK-45239 Project: Spark Issue Type: New

[jira] [Updated] (SPARK-45207) Implement Complete Error Reconstruction for Scala Client

2023-09-20 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-45207: -- Summary: Implement Complete Error Reconstruction for Scala Client (was: Implement Error Enrichment

[jira] [Updated] (SPARK-45207) Implement Error Enrichment for Scala Client

2023-09-19 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-45207: -- Summary: Implement Error Enrichment for Scala Client (was: Implement FetchErrorDetails RPC) >

[jira] [Updated] (SPARK-44622) Implement FetchErrorDetails RPC

2023-09-19 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-44622: -- Summary: Implement FetchErrorDetails RPC (was: Implement error enrichment and JVM stacktrace) >

[jira] [Created] (SPARK-45207) Implement FetchErrorDetails RPC

2023-09-18 Thread Yihong He (Jira)
Yihong He created SPARK-45207: - Summary: Implement FetchErrorDetails RPC Key: SPARK-45207 URL: https://issues.apache.org/jira/browse/SPARK-45207 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-45050) Improve error message for UNKNOWN io.grpc.StatusRuntimeException

2023-09-01 Thread Yihong He (Jira)
Yihong He created SPARK-45050: - Summary: Improve error message for UNKNOWN io.grpc.StatusRuntimeException Key: SPARK-45050 URL: https://issues.apache.org/jira/browse/SPARK-45050 Project: Spark

[jira] [Commented] (SPARK-43299) JVM Client throw StreamingQueryException when error handling is implemented

2023-08-30 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17760620#comment-17760620 ] Yihong He commented on SPARK-43299: --- [~hvanhovell] Thanks for the reminder! I will make sure it works.

[jira] [Resolved] (SPARK-44874) Handle unrecognizable exceptions

2023-08-19 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He resolved SPARK-44874. --- Resolution: Duplicate > Handle unrecognizable exceptions > > >

[jira] [Updated] (SPARK-44874) Handle unrecognizable exceptions

2023-08-18 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-44874: -- Summary: Handle unrecognizable exceptions (was: Implement SparkConnectServerException) > Handle

[jira] [Updated] (SPARK-44874) Implement SparkConnectServerException

2023-08-18 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-44874: -- Summary: Implement SparkConnectServerException (was: Implement SparkConnectGrpcException) >

[jira] [Updated] (SPARK-44874) Implement SparkConnectGrpcException

2023-08-18 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-44874: -- Summary: Implement SparkConnectGrpcException (was: Implement SparkConnectServerException) >

[jira] [Created] (SPARK-44874) Implement SparkConnectServerException

2023-08-18 Thread Yihong He (Jira)
Yihong He created SPARK-44874: - Summary: Implement SparkConnectServerException Key: SPARK-44874 URL: https://issues.apache.org/jira/browse/SPARK-44874 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-44622) Implement Error enrichment and JVM stacktrace

2023-08-13 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-44622: -- Summary: Implement Error enrichment and JVM stacktrace (was: Add GetErrorInfo RPC) > Implement

[jira] [Updated] (SPARK-44622) Implement error enrichment and JVM stacktrace

2023-08-13 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-44622: -- Summary: Implement error enrichment and JVM stacktrace (was: Implement Error enrichment and JVM

[jira] [Created] (SPARK-44786) Convert common Spark exceptions

2023-08-12 Thread Yihong He (Jira)
Yihong He created SPARK-44786: - Summary: Convert common Spark exceptions Key: SPARK-44786 URL: https://issues.apache.org/jira/browse/SPARK-44786 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-44785) Convert common alreadyExistsExceptions and noSuchExceptions

2023-08-12 Thread Yihong He (Jira)
Yihong He created SPARK-44785: - Summary: Convert common alreadyExistsExceptions and noSuchExceptions Key: SPARK-44785 URL: https://issues.apache.org/jira/browse/SPARK-44785 Project: Spark Issue

[jira] [Created] (SPARK-44744) Move DS v2 API to sql/api module

2023-08-09 Thread Yihong He (Jira)
Yihong He created SPARK-44744: - Summary: Move DS v2 API to sql/api module Key: SPARK-44744 URL: https://issues.apache.org/jira/browse/SPARK-44744 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-44622) Add GetErrorInfo RPC

2023-08-06 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-44622: -- Summary: Add GetErrorInfo RPC (was: Add GetDebugInfo RPC) > Add GetErrorInfo RPC >

[jira] [Created] (SPARK-44691) Move Subclasses of Analysis to sql/api

2023-08-06 Thread Yihong He (Jira)
Yihong He created SPARK-44691: - Summary: Move Subclasses of Analysis to sql/api Key: SPARK-44691 URL: https://issues.apache.org/jira/browse/SPARK-44691 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-44622) Add GetDebugInfo RPC

2023-08-01 Thread Yihong He (Jira)
Yihong He created SPARK-44622: - Summary: Add GetDebugInfo RPC Key: SPARK-44622 URL: https://issues.apache.org/jira/browse/SPARK-44622 Project: Spark Issue Type: New Feature Components:

[jira] [Created] (SPARK-44590) Remove the arrow batch record limit for SqlCommandResult

2023-07-28 Thread Yihong He (Jira)
Yihong He created SPARK-44590: - Summary: Remove the arrow batch record limit for SqlCommandResult Key: SPARK-44590 URL: https://issues.apache.org/jira/browse/SPARK-44590 Project: Spark Issue

[jira] [Created] (SPARK-44587) Increase Protobuf Marshaller Recursion Limit

2023-07-28 Thread Yihong He (Jira)
Yihong He created SPARK-44587: - Summary: Increase Protobuf Marshaller Recursion Limit Key: SPARK-44587 URL: https://issues.apache.org/jira/browse/SPARK-44587 Project: Spark Issue Type: New

[jira] [Created] (SPARK-44575) Implement Error Translation

2023-07-27 Thread Yihong He (Jira)
Yihong He created SPARK-44575: - Summary: Implement Error Translation Key: SPARK-44575 URL: https://issues.apache.org/jira/browse/SPARK-44575 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-44278) Implement a GRPC server interceptor that cleans up thread local properties

2023-07-03 Thread Yihong He (Jira)
Yihong He created SPARK-44278: - Summary: Implement a GRPC server interceptor that cleans up thread local properties Key: SPARK-44278 URL: https://issues.apache.org/jira/browse/SPARK-44278 Project: Spark

[jira] [Updated] (SPARK-44193) Implement GRPC exceptions interception for conversion

2023-06-26 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-44193: -- Summary: Implement GRPC exceptions interception for conversion (was: Implement GRPC exceptions

[jira] [Created] (SPARK-44193) Implement GRPC exceptions interception

2023-06-26 Thread Yihong He (Jira)
Yihong He created SPARK-44193: - Summary: Implement GRPC exceptions interception Key: SPARK-44193 URL: https://issues.apache.org/jira/browse/SPARK-44193 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-38556) Disable Pandas usage logging for method calls inside @contextmanager functions

2022-03-15 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-38556: -- Summary: Disable Pandas usage logging for method calls inside @contextmanager functions (was:

[jira] [Updated] (SPARK-38556) Disable Pandas usage logging for calls inside @contextmanager functions

2022-03-15 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-38556: -- Description: Currently, calls inside @contextmanager functions are treated as external for *with*

[jira] [Created] (SPARK-38556) Disable Pandas usage logging for calls inside @contextmanager functions

2022-03-15 Thread Yihong He (Jira)
Yihong He created SPARK-38556: - Summary: Disable Pandas usage logging for calls inside @contextmanager functions Key: SPARK-38556 URL: https://issues.apache.org/jira/browse/SPARK-38556 Project: Spark

[jira] [Updated] (SPARK-38484) Move usage logging instrumentation util functions from pandas module to pyspark.util module

2022-03-09 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-38484: -- Description: It will be helpful to attach the usage logger to other modules (e.g. sql) besides Pandas

[jira] [Updated] (SPARK-38484) Move usage logging instrumentation util functions from pandas module to pyspark.util module

2022-03-09 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-38484: -- Description: It will be helpful to attach the usage logger to other modules (e.g. sql) besides Pandas

[jira] [Created] (SPARK-38484) Move usage logging instrumentation util functions from pandas module to pyspark.util module

2022-03-09 Thread Yihong He (Jira)
Yihong He created SPARK-38484: - Summary: Move usage logging instrumentation util functions from pandas module to pyspark.util module Key: SPARK-38484 URL: https://issues.apache.org/jira/browse/SPARK-38484

[jira] [Updated] (SPARK-38353) Instrument __enter__ and __exit__ magic methods for Pandas module

2022-03-02 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-38353: -- Description: Create the ticket since instrumenting {_}{{_}}enter{{_}}{_} and {_}{{_}}exit{{_}}{_}

[jira] [Updated] (SPARK-38353) Instrument __enter__ and __exit__ magic methods for Pandas module

2022-03-02 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-38353: -- Description: Create the ticket since instrumenting {_}{{_}}enter{{_}}{_} and {_}{{_}}exit{{_}}{_}

[jira] [Updated] (SPARK-38353) Instrument __enter__ and __exit__ magic methods for Pandas module

2022-03-02 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-38353: -- Description: Create the ticket since instrumenting _{_}enter{_}_ and _{_}exit{_}_ magic methods for

[jira] [Updated] (SPARK-38353) Instrument __enter__ and __exit__ magic methods for Pandas module

2022-03-02 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-38353: -- Description: For example, for the following code:   {code:java} pdf = pd.DataFrame( [(0.2, 0.3),

[jira] [Updated] (SPARK-38353) Instrument __enter__ and __exit__ magic methods for Pandas module

2022-03-02 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-38353: -- Description: For example, for the following code:   {code:java} pdf = pd.DataFrame( [(0.2, 0.3),

[jira] [Reopened] (SPARK-38353) Instrument __enter__ and __exit__ magic methods for Pandas module

2022-02-28 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He reopened SPARK-38353: --- > Instrument __enter__ and __exit__ magic methods for Pandas module >

[jira] [Resolved] (SPARK-38353) Instrument __enter__ and __exit__ magic methods for Pandas module

2022-02-28 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He resolved SPARK-38353. --- Resolution: Abandoned > Instrument __enter__ and __exit__ magic methods for Pandas module >

[jira] [Created] (SPARK-38353) Instrument __enter__ and __exit__ magic methods for Pandas module

2022-02-28 Thread Yihong He (Jira)
Yihong He created SPARK-38353: - Summary: Instrument __enter__ and __exit__ magic methods for Pandas module Key: SPARK-38353 URL: https://issues.apache.org/jira/browse/SPARK-38353 Project: Spark