[jira] [Updated] (SPARK-46275) Protobuf: Permissive mode should return null rather than struct with null fields

2023-12-07 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated SPARK-46275: - Fix Version/s: 3.4.3 > Protobuf: Permissive mode should return null rather than struct with

[jira] [Updated] (SPARK-46275) Protobuf: Permissive mode should return null rather than struct with null fields

2023-12-07 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated SPARK-46275: - Affects Version/s: (was: 3.5.0) > Protobuf: Permissive mode should return null rather than

[jira] [Updated] (SPARK-46275) Protobuf: Permissive mode should return null rather than struct with null fields

2023-12-07 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated SPARK-46275: - Affects Version/s: 3.4.0 > Protobuf: Permissive mode should return null rather than struct with

[jira] [Updated] (SPARK-46275) Protobuf: Permissive mode should return null rather than struct with null fields

2023-12-05 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated SPARK-46275: - Description: Consider a protobuf with two fields {{message Person { string name = 1; int id =

[jira] [Commented] (SPARK-46275) Protobuf: Permissive mode should return null rather than struct with null fields

2023-12-05 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793453#comment-17793453 ] Raghu Angadi commented on SPARK-46275: -- Context from [~gengliang] about `from_avro()` behavior:

[jira] [Updated] (SPARK-46275) Protobuf: Permissive mode should return null rather than struct with null fields

2023-12-05 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated SPARK-46275: - Description: Consider a protobuf with two fields {{message Person { string name = 1; int id =

[jira] [Created] (SPARK-46275) Protobuf: Permissive mode should return null rather than struct with null fields

2023-12-05 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-46275: Summary: Protobuf: Permissive mode should return null rather than struct with null fields Key: SPARK-46275 URL: https://issues.apache.org/jira/browse/SPARK-46275

[jira] [Updated] (SPARK-45957) SQL on streaming Temp view fails

2023-11-16 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated SPARK-45957: - Description: The following code fails in the last step with Spark Connect. The root cause is

[jira] [Created] (SPARK-45957) SQL on streaming Temp view fails

2023-11-16 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-45957: Summary: SQL on streaming Temp view fails Key: SPARK-45957 URL: https://issues.apache.org/jira/browse/SPARK-45957 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-45588) Minor scaladoc improvement in StreamingForeachBatchHelper

2023-10-17 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-45588: Summary: Minor scaladoc improvement in StreamingForeachBatchHelper Key: SPARK-45588 URL: https://issues.apache.org/jira/browse/SPARK-45588 Project: Spark

[jira] [Created] (SPARK-45245) PythonWorkerFactory create gets stuck in case of worker failures

2023-09-20 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-45245: Summary: PythonWorkerFactory create gets stuck in case of worker failures Key: SPARK-45245 URL: https://issues.apache.org/jira/browse/SPARK-45245 Project: Spark

[jira] [Created] (SPARK-45138) Define a new error class and apply for the case where checkpointing state to DFS fails

2023-09-12 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-45138: Summary: Define a new error class and apply for the case where checkpointing state to DFS fails Key: SPARK-45138 URL: https://issues.apache.org/jira/browse/SPARK-45138

[jira] [Resolved] (SPARK-44460) Pass user auth credential to Python workers for foreachBatch and listener

2023-08-21 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi resolved SPARK-44460. -- Resolution: Won't Fix Not an issue in Apache Spark.  > Pass user auth credential to Python

[jira] [Updated] (SPARK-44462) Fix the session passed to foreachBatch.

2023-08-16 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated SPARK-44462: - Description: foreachBatch() in Connect uses initial session used while starting the streaming

[jira] [Commented] (SPARK-44462) Fix the session passed to foreachBatch.

2023-08-16 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755296#comment-17755296 ] Raghu Angadi commented on SPARK-44462: -- I will work on this. > Fix the session passed to

[jira] [Commented] (SPARK-42962) Allow access to stopped streaming queries

2023-08-16 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755295#comment-17755295 ] Raghu Angadi commented on SPARK-42962: -- This is done. Stopped queries are accessible for an hour.

[jira] [Closed] (SPARK-43144) Scala: DataStreamReader table() API

2023-08-16 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi closed SPARK-43144. > Scala: DataStreamReader table() API > --- > > Key:

[jira] [Commented] (SPARK-44460) Pass user auth credential to Python workers for foreachBatch and listener

2023-08-16 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755293#comment-17755293 ] Raghu Angadi commented on SPARK-44460: -- [~WweiL] I think you looked into this issue. could you add

[jira] [Updated] (SPARK-44764) Streaming Python process management improvement

2023-08-16 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated SPARK-44764: - Summary: Streaming Python process management improvement (was: Streaming process improvement)

[jira] [Created] (SPARK-44463) Improve error handling in Connect foreachBatch worker.

2023-07-17 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-44463: Summary: Improve error handling in Connect foreachBatch worker. Key: SPARK-44463 URL: https://issues.apache.org/jira/browse/SPARK-44463 Project: Spark Issue

[jira] [Created] (SPARK-44462) Fix the session passed to foreachBatch.

2023-07-17 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-44462: Summary: Fix the session passed to foreachBatch. Key: SPARK-44462 URL: https://issues.apache.org/jira/browse/SPARK-44462 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-44461) Enable Process Isolation for streaming python worker

2023-07-17 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-44461: Summary: Enable Process Isolation for streaming python worker Key: SPARK-44461 URL: https://issues.apache.org/jira/browse/SPARK-44461 Project: Spark Issue

[jira] [Created] (SPARK-44460) Pass user auth credential to Python workers for foreachBatch and listener

2023-07-17 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-44460: Summary: Pass user auth credential to Python workers for foreachBatch and listener Key: SPARK-44460 URL: https://issues.apache.org/jira/browse/SPARK-44460 Project:

[jira] [Resolved] (SPARK-43054) Support foreach() in streaming spark connect

2023-07-14 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi resolved SPARK-43054. -- Resolution: Fixed > Support foreach() in streaming spark connect >

[jira] (SPARK-43054) Support foreach() in streaming spark connect

2023-07-14 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43054 ] Raghu Angadi deleted comment on SPARK-43054: -- was (Author: rangadi): This is for python. Keep it open. > Support foreach() in streaming spark connect >

[jira] [Reopened] (SPARK-42944) Support Python foreachBatch() in streaming spark connect

2023-07-14 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi reopened SPARK-42944: -- > Support Python foreachBatch() in streaming spark connect >

[jira] [Updated] (SPARK-42944) Support Python foreachBatch() in streaming spark connect

2023-07-14 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated SPARK-42944: - Summary: Support Python foreachBatch() in streaming spark connect (was: Support foreachBatch()

[jira] [Reopened] (SPARK-43054) Support foreach() in streaming spark connect

2023-07-14 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi reopened SPARK-43054: -- This is for python. Keep it open. > Support foreach() in streaming spark connect >

[jira] [Resolved] (SPARK-42944) Support foreachBatch() in streaming spark connect

2023-07-14 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi resolved SPARK-42944. -- Resolution: Fixed Handled in other tickets.  > Support foreachBatch() in streaming spark

[jira] [Created] (SPARK-44435) Add more tests for Python foreachBatch and StreamingQueryListener

2023-07-14 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-44435: Summary: Add more tests for Python foreachBatch and StreamingQueryListener Key: SPARK-44435 URL: https://issues.apache.org/jira/browse/SPARK-44435 Project: Spark

[jira] [Created] (SPARK-44434) Add more tests for Scala foreachBatch and streaming listers

2023-07-14 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-44434: Summary: Add more tests for Scala foreachBatch and streaming listers Key: SPARK-44434 URL: https://issues.apache.org/jira/browse/SPARK-44434 Project: Spark

[jira] [Created] (SPARK-44433) Implement termination of Python process for foreachBatch & streaming listener

2023-07-14 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-44433: Summary: Implement termination of Python process for foreachBatch & streaming listener Key: SPARK-44433 URL: https://issues.apache.org/jira/browse/SPARK-44433

[jira] [Created] (SPARK-44432) Terminate streaming queries when a session times out in Spark Connect

2023-07-14 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-44432: Summary: Terminate streaming queries when a session times out in Spark Connect Key: SPARK-44432 URL: https://issues.apache.org/jira/browse/SPARK-44432 Project: Spark

[jira] [Updated] (SPARK-44400) Improve Scala StreamingQueryListener to provide users a way to access the Spark session for Spark Connect

2023-07-13 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated SPARK-44400: - Epic Link: SPARK-42938 > Improve Scala StreamingQueryListener to provide users a way to access

[jira] [Created] (SPARK-44398) Scala foreachBatch API in Streaming Spark Connect

2023-07-12 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-44398: Summary: Scala foreachBatch API in Streaming Spark Connect Key: SPARK-44398 URL: https://issues.apache.org/jira/browse/SPARK-44398 Project: Spark Issue

[jira] [Created] (SPARK-44337) Any fields set to Any.getDefaultInstance cause exceptions.

2023-07-07 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-44337: Summary: Any fields set to Any.getDefaultInstance cause exceptions. Key: SPARK-44337 URL: https://issues.apache.org/jira/browse/SPARK-44337 Project: Spark

[jira] [Created] (SPARK-43921) Generate Protobuf descriptor files at build time

2023-06-01 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-43921: Summary: Generate Protobuf descriptor files at build time Key: SPARK-43921 URL: https://issues.apache.org/jira/browse/SPARK-43921 Project: Spark Issue Type:

[jira] [Commented] (SPARK-43646) Maven test failed: `from_protobuf_messageClassName` and `from_protobuf_messageClassName_options`

2023-05-29 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17727284#comment-17727284 ] Raghu Angadi commented on SPARK-43646: -- [~LuciferYang] how are CI builds passing in github? >

[jira] [Created] (SPARK-43847) Protobuf: Throw structured error when reading descriptor file in Python fails

2023-05-28 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-43847: Summary: Protobuf: Throw structured error when reading descriptor file in Python fails Key: SPARK-43847 URL: https://issues.apache.org/jira/browse/SPARK-43847

[jira] [Created] (SPARK-43799) Protobuf: Support binary descriptor set API in Python

2023-05-25 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-43799: Summary: Protobuf: Support binary descriptor set API in Python Key: SPARK-43799 URL: https://issues.apache.org/jira/browse/SPARK-43799 Project: Spark Issue

[jira] [Created] (SPARK-43578) Protobuf : Use broadcast to distribute descriptor file content

2023-05-18 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-43578: Summary: Protobuf : Use broadcast to distribute descriptor file content Key: SPARK-43578 URL: https://issues.apache.org/jira/browse/SPARK-43578 Project: Spark

[jira] [Created] (SPARK-43530) Protobuf: Read descriptor file only once at the compile time

2023-05-16 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-43530: Summary: Protobuf: Read descriptor file only once at the compile time Key: SPARK-43530 URL: https://issues.apache.org/jira/browse/SPARK-43530 Project: Spark

[jira] [Updated] (SPARK-43312) Protobuf: Allow converting Any fields to JSON

2023-04-27 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated SPARK-43312: - Summary: Protobuf: Allow converting Any fields to JSON (was: Protobfu: Allow converting Any

[jira] [Created] (SPARK-43312) Protobfu: Allow converting Any fields to JSON

2023-04-27 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-43312: Summary: Protobfu: Allow converting Any fields to JSON Key: SPARK-43312 URL: https://issues.apache.org/jira/browse/SPARK-43312 Project: Spark Issue Type:

[jira] [Created] (SPARK-43144) Scala: DataStreamReader table() API

2023-04-14 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-43144: Summary: Scala: DataStreamReader table() API Key: SPARK-43144 URL: https://issues.apache.org/jira/browse/SPARK-43144 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-43143) Scala: Add StreamingQuery awaitTermination() API

2023-04-14 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-43143: Summary: Scala: Add StreamingQuery awaitTermination() API Key: SPARK-43143 URL: https://issues.apache.org/jira/browse/SPARK-43143 Project: Spark Issue Type:

[jira] [Created] (SPARK-43134) Add streaming query exception API in Scala

2023-04-13 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-43134: Summary: Add streaming query exception API in Scala Key: SPARK-43134 URL: https://issues.apache.org/jira/browse/SPARK-43134 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-43132) Add foreach streaming API in Python

2023-04-13 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-43132: Summary: Add foreach streaming API in Python Key: SPARK-43132 URL: https://issues.apache.org/jira/browse/SPARK-43132 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-43133) Add foreach streaming API in Scala

2023-04-13 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-43133: Summary: Add foreach streaming API in Scala Key: SPARK-43133 URL: https://issues.apache.org/jira/browse/SPARK-43133 Project: Spark Issue Type: Task

[jira] [Updated] (SPARK-43129) Scala Core API for Streaming Spark Connect

2023-04-13 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated SPARK-43129: - Summary: Scala Core API for Streaming Spark Connect (was: Scala MVP for Streaming Spark

[jira] [Created] (SPARK-43129) Scala MVP for Streaming Spark Connect

2023-04-13 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-43129: Summary: Scala MVP for Streaming Spark Connect Key: SPARK-43129 URL: https://issues.apache.org/jira/browse/SPARK-43129 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-43128) Streaming progress struct (especially in Scala)

2023-04-13 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-43128: Summary: Streaming progress struct (especially in Scala) Key: SPARK-43128 URL: https://issues.apache.org/jira/browse/SPARK-43128 Project: Spark Issue Type:

[jira] [Created] (SPARK-43084) Add Python state API (applyInPandasWithState) and verify UDFs

2023-04-10 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-43084: Summary: Add Python state API (applyInPandasWithState) and verify UDFs Key: SPARK-43084 URL: https://issues.apache.org/jira/browse/SPARK-43084 Project: Spark

[jira] [Created] (SPARK-43032) Add StreamingQueryManager API

2023-04-04 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-43032: Summary: Add StreamingQueryManager API Key: SPARK-43032 URL: https://issues.apache.org/jira/browse/SPARK-43032 Project: Spark Issue Type: Task

[jira] [Commented] (SPARK-43031) Enable tests for Python streaming spark-connect

2023-04-04 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17708519#comment-17708519 ] Raghu Angadi commented on SPARK-43031: -- cc: [~WweiL]  > Enable tests for Python streaming

[jira] [Created] (SPARK-43031) Enable tests for Python streaming spark-connect

2023-04-04 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-43031: Summary: Enable tests for Python streaming spark-connect Key: SPARK-43031 URL: https://issues.apache.org/jira/browse/SPARK-43031 Project: Spark Issue Type:

[jira] [Created] (SPARK-42962) Allow access to stopped streaming queries

2023-03-29 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-42962: Summary: Allow access to stopped streaming queries Key: SPARK-42962 URL: https://issues.apache.org/jira/browse/SPARK-42962 Project: Spark Issue Type: Task

[jira] [Updated] (SPARK-42961) Add remaining Streaming Query commands like avait_termination()

2023-03-29 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated SPARK-42961: - Epic Link: (was: SPARK-42938) > Add remaining Streaming Query commands like

[jira] [Commented] (SPARK-42959) Add remaining Streaming Query commands like avait_termination()

2023-03-29 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17706226#comment-17706226 ] Raghu Angadi commented on SPARK-42959: -- This was a duplicate. > Add remaining Streaming Query

[jira] [Updated] (SPARK-42959) Add remaining Streaming Query commands like avait_termination()

2023-03-29 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated SPARK-42959: - Epic Link: (was: SPARK-42938) > Add remaining Streaming Query commands like

[jira] [Resolved] (SPARK-42961) Add remaining Streaming Query commands like avait_termination()

2023-03-29 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi resolved SPARK-42961. -- Resolution: Duplicate > Add remaining Streaming Query commands like avait_termination() >

[jira] [Created] (SPARK-42961) Add remaining Streaming Query commands like avait_termination()

2023-03-29 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-42961: Summary: Add remaining Streaming Query commands like avait_termination() Key: SPARK-42961 URL: https://issues.apache.org/jira/browse/SPARK-42961 Project: Spark

[jira] [Resolved] (SPARK-42959) Add remaining Streaming Query commands like avait_termination()

2023-03-29 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi resolved SPARK-42959. -- Resolution: Duplicate > Add remaining Streaming Query commands like avait_termination() >

[jira] [Created] (SPARK-42960) Add remaining Streaming Query commands like avait_termination()

2023-03-29 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-42960: Summary: Add remaining Streaming Query commands like avait_termination() Key: SPARK-42960 URL: https://issues.apache.org/jira/browse/SPARK-42960 Project: Spark

[jira] [Created] (SPARK-42959) Add remaining Streaming Query commands like avait_termination()

2023-03-29 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-42959: Summary: Add remaining Streaming Query commands like avait_termination() Key: SPARK-42959 URL: https://issues.apache.org/jira/browse/SPARK-42959 Project: Spark

[jira] [Created] (SPARK-42944) Support foreachBatch() in streaming spark connect

2023-03-27 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-42944: Summary: Support foreachBatch() in streaming spark connect Key: SPARK-42944 URL: https://issues.apache.org/jira/browse/SPARK-42944 Project: Spark Issue

[jira] [Created] (SPARK-42941) Add support for streaming listener.

2023-03-27 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-42941: Summary: Add support for streaming listener. Key: SPARK-42941 URL: https://issues.apache.org/jira/browse/SPARK-42941 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-42940) Session management support streaming connect

2023-03-27 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-42940: Summary: Session management support streaming connect Key: SPARK-42940 URL: https://issues.apache.org/jira/browse/SPARK-42940 Project: Spark Issue Type:

[jira] [Created] (SPARK-42939) Streaming connect core API

2023-03-27 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-42939: Summary: Streaming connect core API Key: SPARK-42939 URL: https://issues.apache.org/jira/browse/SPARK-42939 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-42938) Structured Streaming with Spark Connect

2023-03-27 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-42938: Summary: Structured Streaming with Spark Connect Key: SPARK-42938 URL: https://issues.apache.org/jira/browse/SPARK-42938 Project: Spark Issue Type: Epic

[jira] [Created] (SPARK-42721) Add an Interceptor to log RPCs in connect-server

2023-03-08 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-42721: Summary: Add an Interceptor to log RPCs in connect-server Key: SPARK-42721 URL: https://issues.apache.org/jira/browse/SPARK-42721 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-40659) Schema evolution for protobuf (and Avro too?)

2023-02-27 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi resolved SPARK-40659. -- Resolution: Won't Do > Schema evolution for protobuf (and Avro too?) >

[jira] [Updated] (SPARK-40848) Protobuf: Generate descriptor files at build time

2023-02-27 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated SPARK-40848: - Epic Link: SPARK-42606 (was: SPARK-40653) > Protobuf: Generate descriptor files at build time

[jira] [Resolved] (SPARK-40658) Protobuf v2 & v3 support

2023-02-27 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi resolved SPARK-40658. -- Resolution: Fixed > Protobuf v2 & v3 support > > >

[jira] [Created] (SPARK-42606) Improvements to built in Protobuf functions

2023-02-27 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-42606: Summary: Improvements to built in Protobuf functions Key: SPARK-42606 URL: https://issues.apache.org/jira/browse/SPARK-42606 Project: Spark Issue Type: Epic

[jira] [Commented] (SPARK-42540) Add more test case(s) for longer chain of multiple stateful operators

2023-02-23 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17692942#comment-17692942 ] Raghu Angadi commented on SPARK-42540: -- I am interested in working on this. Not sure about ETA. >

[jira] [Reopened] (SPARK-42406) [PROTOBUF] Recursive field handling is incompatible with delta

2023-02-23 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi reopened SPARK-42406: -- The main for still to be merged. > [PROTOBUF] Recursive field handling is incompatible with

[jira] [Resolved] (SPARK-40653) Protobuf Support in Structured Streaming

2023-02-15 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi resolved SPARK-40653. -- Fix Version/s: 3.4.0 Resolution: Fixed Protobuf functions have been in use for couple

[jira] [Commented] (SPARK-42406) [PROTOBUF] Recursive field handling is incompatible with delta

2023-02-14 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688752#comment-17688752 ] Raghu Angadi commented on SPARK-42406: -- Thanks for merging

[jira] [Updated] (SPARK-42406) [PROTOBUF] Recursive field handling is incompatible with delta

2023-02-14 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated SPARK-42406: - Description: Protobuf deserializer (`from_protobuf()` function()) optionally supports

[jira] [Commented] (SPARK-42406) [PROTOBUF] Recursive field handling is incompatible with delta

2023-02-13 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17688033#comment-17688033 ] Raghu Angadi commented on SPARK-42406: -- I am working on a fix. > [PROTOBUF] Recursive field

[jira] [Commented] (SPARK-42406) [PROTOBUF] Recursive field handling is incompatible with delta

2023-02-11 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17687489#comment-17687489 ] Raghu Angadi commented on SPARK-42406: -- cc: [~sanysand...@gmail.com] PTAL. > [PROTOBUF] Recursive

[jira] [Updated] (SPARK-42406) [PROTOBUF] Recursive field handling is incompatible with delta

2023-02-11 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated SPARK-42406: - Description: Protobuf deserializer (`from_protobuf()` function()) optionally supports

[jira] [Created] (SPARK-42406) [PROTOBUF] Recursive field handling is incompatible with delta

2023-02-11 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-42406: Summary: [PROTOBUF] Recursive field handling is incompatible with delta Key: SPARK-42406 URL: https://issues.apache.org/jira/browse/SPARK-42406 Project: Spark

[jira] [Updated] (SPARK-42056) Add missing options for Protobuf functions.

2023-01-13 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated SPARK-42056: - Summary: Add missing options for Protobuf functions. (was: Add missing options arg for

[jira] [Created] (SPARK-42057) Avoid losing exception info in Protobuf errors

2023-01-13 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-42057: Summary: Avoid losing exception info in Protobuf errors Key: SPARK-42057 URL: https://issues.apache.org/jira/browse/SPARK-42057 Project: Spark Issue Type:

[jira] [Created] (SPARK-42056) Add missing options arg for Protobuf functions.

2023-01-13 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-42056: Summary: Add missing options arg for Protobuf functions. Key: SPARK-42056 URL: https://issues.apache.org/jira/browse/SPARK-42056 Project: Spark Issue Type:

[jira] [Commented] (SPARK-41015) Failure of ProtobufCatalystDataConversionSuite.scala

2022-11-04 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17629204#comment-17629204 ] Raghu Angadi commented on SPARK-41015: -- Thanks for filing this [~maxgekk] . cc:

[jira] [Commented] (SPARK-40658) Protobuf v2 & v3 support

2022-10-20 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621315#comment-17621315 ] Raghu Angadi commented on SPARK-40658: -- [~mposdev21] PTAL at my changes in a temporary PR :

[jira] [Resolved] (SPARK-40656) Schema-registry support for Protobuf format

2022-10-19 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi resolved SPARK-40656. -- Resolution: Won't Do > Schema-registry support for Protobuf format >

[jira] [Commented] (SPARK-40659) Schema evolution for protobuf (and Avro too?)

2022-10-19 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17620661#comment-17620661 ] Raghu Angadi commented on SPARK-40659: -- Right, I don't think that is the reason. Databricks might

[jira] [Commented] (SPARK-40658) Protobuf v2 & v3 support

2022-10-19 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17620659#comment-17620659 ] Raghu Angadi commented on SPARK-40658: -- Thats awesome! Lets look at both and merge them. >

[jira] [Commented] (SPARK-40658) Protobuf v2 & v3 support

2022-10-19 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17620655#comment-17620655 ] Raghu Angadi commented on SPARK-40658: -- [~mposdev21] more tests are good. Lets add them. In my

[jira] [Commented] (SPARK-40659) Schema evolution for protobuf (and Avro too?)

2022-10-19 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17620557#comment-17620557 ] Raghu Angadi commented on SPARK-40659: -- Schema-support for Avro is not in open-source Spark. When

[jira] [Commented] (SPARK-40656) Schema-registry support for Protobuf format

2022-10-19 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17620556#comment-17620556 ] Raghu Angadi commented on SPARK-40656: -- Schema-support for Avro is not here. I will close this.

[jira] [Commented] (SPARK-40777) Use error classes for Protobuf exceptions

2022-10-19 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17620553#comment-17620553 ] Raghu Angadi commented on SPARK-40777: -- I filed a separate ticket SPARK-40848 for generating

[jira] [Commented] (SPARK-40658) Protobuf v2 & v3 support

2022-10-19 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17620551#comment-17620551 ] Raghu Angadi commented on SPARK-40658: -- [~sanysand...@gmail.com] , [~mposdev21] I am working on

[jira] [Created] (SPARK-40848) Protobuf: Generate descriptor files at build time

2022-10-19 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-40848: Summary: Protobuf: Generate descriptor files at build time Key: SPARK-40848 URL: https://issues.apache.org/jira/browse/SPARK-40848 Project: Spark Issue

[jira] [Updated] (SPARK-40776) Add documentation (similar to Avro functions).

2022-10-13 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated SPARK-40776: - Summary: Add documentation (similar to Avro functions). (was: Add documentation (similar Avro

[jira] [Created] (SPARK-40777) Use error classes for Protobuf exceptions

2022-10-12 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-40777: Summary: Use error classes for Protobuf exceptions Key: SPARK-40777 URL: https://issues.apache.org/jira/browse/SPARK-40777 Project: Spark Issue Type:

  1   2   >