[jira] [Resolved] (SPARK-30533) Add classes to represent Java Regressors and RegressionModels

[ https://issues.apache.org/jira/browse/SPARK-30533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-30533. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27241

[jira] [Updated] (SPARK-30533) Add classes to represent Java Regressors and RegressionModels

[ https://issues.apache.org/jira/browse/SPARK-30533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-30533: - Priority: Minor (was: Major) > Add classes to represent Java Regressors and RegressionModels >

[jira] [Assigned] (SPARK-30533) Add classes to represent Java Regressors and RegressionModels

[ https://issues.apache.org/jira/browse/SPARK-30533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-30533: Assignee: Maciej Szymkiewicz > Add classes to represent Java Regressors and

[jira] [Updated] (SPARK-25993) Add test cases for CREATE EXTERNAL TABLE with subdirectories

[ https://issues.apache.org/jira/browse/SPARK-25993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25993: -- Issue Type: Improvement (was: Bug) > Add test cases for CREATE EXTERNAL TABLE with

[jira] [Updated] (SPARK-25993) Add test cases for CREATE EXTERNAL TABLE with subdirectories

[ https://issues.apache.org/jira/browse/SPARK-25993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25993: -- Summary: Add test cases for CREATE EXTERNAL TABLE with subdirectories (was: Add test cases

[jira] [Updated] (SPARK-25993) Add test cases for CREATE EXTERNAL TABLE with subdirectories

[ https://issues.apache.org/jira/browse/SPARK-25993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25993: -- Labels: (was: starter) > Add test cases for CREATE EXTERNAL TABLE with subdirectories >

[jira] [Assigned] (SPARK-25993) Add test cases for resolution of ORC table location

[ https://issues.apache.org/jira/browse/SPARK-25993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25993: - Assignee: kevin yu > Add test cases for resolution of ORC table location >

[jira] [Resolved] (SPARK-25993) Add test cases for resolution of ORC table location

[ https://issues.apache.org/jira/browse/SPARK-25993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25993. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27130

[jira] [Comment Edited] (SPARK-27249) Developers API for Transformers beyond UnaryTransformer

[ https://issues.apache.org/jira/browse/SPARK-27249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018329#comment-17018329 ] Nick Afshartous edited comment on SPARK-27249 at 1/17/20 9:29 PM: --

[jira] [Commented] (SPARK-27249) Developers API for Transformers beyond UnaryTransformer

[ https://issues.apache.org/jira/browse/SPARK-27249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018329#comment-17018329 ] Nick Afshartous commented on SPARK-27249: - [~enrush] Hi Everett, The {{Dataset}} API has an

[jira] [Comment Edited] (SPARK-27249) Developers API for Transformers beyond UnaryTransformer

[ https://issues.apache.org/jira/browse/SPARK-27249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018329#comment-17018329 ] Nick Afshartous edited comment on SPARK-27249 at 1/17/20 9:28 PM: --

[jira] [Updated] (SPARK-27249) Developers API for Transformers beyond UnaryTransformer

[ https://issues.apache.org/jira/browse/SPARK-27249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Afshartous updated SPARK-27249: Attachment: Screen Shot 2020-01-17 at 4.20.57 PM.png > Developers API for Transformers

[jira] [Commented] (SPARK-30557) Add public documentation for SPARK_SUBMIT_OPTS

[ https://issues.apache.org/jira/browse/SPARK-30557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018256#comment-17018256 ] Marcelo Masiero Vanzin commented on SPARK-30557: I don't exactly remember what that

[jira] [Commented] (SPARK-30557) Add public documentation for SPARK_SUBMIT_OPTS

[ https://issues.apache.org/jira/browse/SPARK-30557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018254#comment-17018254 ] Nicholas Chammas commented on SPARK-30557: -- [~vanzin] - Do you know if this is something we

[jira] [Commented] (SPARK-27868) Better document shuffle / RPC listen backlog

[ https://issues.apache.org/jira/browse/SPARK-27868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018242#comment-17018242 ] Dongjoon Hyun commented on SPARK-27868: --- I see. Thank you, [~vanzin]. > Better document shuffle /

[jira] [Resolved] (SPARK-29876) Delete/archive file source completed files in separate thread

[ https://issues.apache.org/jira/browse/SPARK-29876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-29876. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-29876) Delete/archive file source completed files in separate thread

[ https://issues.apache.org/jira/browse/SPARK-29876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-29876: -- Assignee: Gabor Somogyi > Delete/archive file source completed files

[jira] [Created] (SPARK-30557) Add public documentation for SPARK_SUBMIT_OPTS

Nicholas Chammas created SPARK-30557: Summary: Add public documentation for SPARK_SUBMIT_OPTS Key: SPARK-30557 URL: https://issues.apache.org/jira/browse/SPARK-30557 Project: Spark Issue

[jira] [Commented] (SPARK-27868) Better document shuffle / RPC listen backlog

[ https://issues.apache.org/jira/browse/SPARK-27868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018219#comment-17018219 ] Marcelo Masiero Vanzin commented on SPARK-27868: It's ok for now, it's done. Hopefully

[jira] [Updated] (SPARK-22590) Broadcast thread propagates the localProperties to task

[ https://issues.apache.org/jira/browse/SPARK-22590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-22590: Affects Version/s: 3.0.0 2.4.4 > Broadcast thread propagates the localProperties

[jira] [Created] (SPARK-30556) SubqueryExec passes local properties to SubqueryExec.executionContext

Ajith S created SPARK-30556: --- Summary: SubqueryExec passes local properties to SubqueryExec.executionContext Key: SPARK-30556 URL: https://issues.apache.org/jira/browse/SPARK-30556 Project: Spark

[jira] [Assigned] (SPARK-30041) Add Codegen Stage Id to Stage DAG visualization in Web UI

[ https://issues.apache.org/jira/browse/SPARK-30041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-30041: --- Assignee: Luca Canali > Add Codegen Stage Id to Stage DAG visualization in Web UI >

[jira] [Resolved] (SPARK-30041) Add Codegen Stage Id to Stage DAG visualization in Web UI

[ https://issues.apache.org/jira/browse/SPARK-30041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30041. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26675

[jira] [Updated] (SPARK-22590) Broadcast thread propagates the localProperties to task

[ https://issues.apache.org/jira/browse/SPARK-22590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-22590: Summary: Broadcast thread propagates the localProperties to task (was: SparkContext's local properties

[jira] [Updated] (SPARK-22590) SparkContext's local properties missing from TaskContext properties

[ https://issues.apache.org/jira/browse/SPARK-22590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-22590: Description: Local properties set via sparkContext are not available as TaskContext properties when

[jira] [Reopened] (SPARK-22590) SparkContext's local properties missing from TaskContext properties

[ https://issues.apache.org/jira/browse/SPARK-22590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S reopened SPARK-22590: - Adding Fix > SparkContext's local properties missing from TaskContext properties >

[jira] [Created] (SPARK-30555) MERGE INTO insert action should only access columns from source table

Wenchen Fan created SPARK-30555: --- Summary: MERGE INTO insert action should only access columns from source table Key: SPARK-30555 URL: https://issues.apache.org/jira/browse/SPARK-30555 Project: Spark

[jira] [Created] (SPARK-30554) Return Iterable from FailureSafeParser.rawParser

Maxim Gekk created SPARK-30554: -- Summary: Return Iterable from FailureSafeParser.rawParser Key: SPARK-30554 URL: https://issues.apache.org/jira/browse/SPARK-30554 Project: Spark Issue Type:

[jira] [Updated] (SPARK-30553) structured-streaming documentation java watermark group by

[ https://issues.apache.org/jira/browse/SPARK-30553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bettermouse updated SPARK-30553: Description:

[jira] [Updated] (SPARK-30553) structured-streaming documentation java watermark group by

[ https://issues.apache.org/jira/browse/SPARK-30553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bettermouse updated SPARK-30553: Priority: Trivial (was: Major) > structured-streaming documentation java watermark group by >

[jira] [Created] (SPARK-30553) structured-streaming documentation java watermark group by

bettermouse created SPARK-30553: --- Summary: structured-streaming documentation java watermark group by Key: SPARK-30553 URL: https://issues.apache.org/jira/browse/SPARK-30553 Project: Spark

[jira] [Assigned] (SPARK-30310) SparkUncaughtExceptionHandler halts running process unexpectedly

[ https://issues.apache.org/jira/browse/SPARK-30310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-30310: Assignee: Tin Hang To > SparkUncaughtExceptionHandler halts running process unexpectedly

[jira] [Resolved] (SPARK-30310) SparkUncaughtExceptionHandler halts running process unexpectedly

[ https://issues.apache.org/jira/browse/SPARK-30310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-30310. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26955

[jira] [Commented] (SPARK-27249) Developers API for Transformers beyond UnaryTransformer

[ https://issues.apache.org/jira/browse/SPARK-27249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018105#comment-17018105 ] Nick Afshartous commented on SPARK-27249: - Thanks Everett, and can someone with permission

[jira] [Updated] (SPARK-30552) Chained spark column expressions with distinct windows specs produce inefficient DAG

[ https://issues.apache.org/jira/browse/SPARK-30552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Franz updated SPARK-30552: -- Environment: python : 3.6.9.final.0 python-bits : 64 OS : Windows OS-release : 10 machine : AMD64

[jira] [Created] (SPARK-30552) Chained spark column expressions with distinct windows specs produce inefficient DAG

Franz created SPARK-30552: - Summary: Chained spark column expressions with distinct windows specs produce inefficient DAG Key: SPARK-30552 URL: https://issues.apache.org/jira/browse/SPARK-30552 Project:

[jira] [Resolved] (SPARK-29306) Executors need to track what ResourceProfile they are created with

[ https://issues.apache.org/jira/browse/SPARK-29306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-29306. --- Fix Version/s: 3.0.0 Resolution: Fixed > Executors need to track what

[jira] [Comment Edited] (SPARK-30542) Two Spark structured streaming jobs cannot write to same base path

[ https://issues.apache.org/jira/browse/SPARK-30542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017820#comment-17017820 ] Sivakumar edited comment on SPARK-30542 at 1/17/20 12:40 PM: - Hi Jungtaek,

[jira] [Issue Comment Deleted] (SPARK-30542) Two Spark structured streaming jobs cannot write to same base path

[ https://issues.apache.org/jira/browse/SPARK-30542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sivakumar updated SPARK-30542: -- Comment: was deleted (was: Hi Jungtaek, I thought this might be a feature that should be added to

[jira] [Comment Edited] (SPARK-30542) Two Spark structured streaming jobs cannot write to same base path

[ https://issues.apache.org/jira/browse/SPARK-30542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017967#comment-17017967 ] Sivakumar edited comment on SPARK-30542 at 1/17/20 12:40 PM: - Hi Jungtaek,

[jira] [Commented] (SPARK-30542) Two Spark structured streaming jobs cannot write to same base path

[ https://issues.apache.org/jira/browse/SPARK-30542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017967#comment-17017967 ] Sivakumar commented on SPARK-30542: --- Hi Jungtaek, I thought this might be a feature that should be

[jira] [Updated] (SPARK-30282) Migrate SHOW TBLPROPERTIES to new framework

[ https://issues.apache.org/jira/browse/SPARK-30282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-30282: Summary: Migrate SHOW TBLPROPERTIES to new framework (was: UnresolvedV2Relation should be

[jira] [Commented] (SPARK-29299) Intermittently getting "Cannot create the managed table error" while creating table from spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-29299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017932#comment-17017932 ] Steve Loughran commented on SPARK-29299: Have you tried using the s3 optimised committer in EMR?

[jira] [Commented] (SPARK-30393) Too much ProvisionedThroughputExceededException while recover from checkpoint

[ https://issues.apache.org/jira/browse/SPARK-30393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017930#comment-17017930 ] Steve Loughran commented on SPARK-30393: {{ProvisionedThroughputExceededException}} means your

[jira] [Commented] (SPARK-30460) Spark checkpoint failing after some run with S3 path

[ https://issues.apache.org/jira/browse/SPARK-30460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017928#comment-17017928 ] Steve Loughran commented on SPARK-30460: bq. I understand spark 3.0 has new committer but as you

[jira] [Created] (SPARK-30551) Disable comparison for interval type

Kent Yao created SPARK-30551: Summary: Disable comparison for interval type Key: SPARK-30551 URL: https://issues.apache.org/jira/browse/SPARK-30551 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-30550) Random pyspark-shell applications being generated

[ https://issues.apache.org/jira/browse/SPARK-30550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ram updated SPARK-30550: Attachment: Screenshot from 2020-01-17 15-43-33.png > Random pyspark-shell applications being generated >

[jira] [Updated] (SPARK-30550) Random pyspark-shell applications being generated

[ https://issues.apache.org/jira/browse/SPARK-30550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ram updated SPARK-30550: Description:   When we submit a particular spark job, this happens. Not sure from where these pyspark-shell

[jira] [Created] (SPARK-30550) Random pyspark-shell applications being generated

Ram created SPARK-30550: --- Summary: Random pyspark-shell applications being generated Key: SPARK-30550 URL: https://issues.apache.org/jira/browse/SPARK-30550 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-30549) Fix the subquery metrics showing issue in UI When enable AQE

[ https://issues.apache.org/jira/browse/SPARK-30549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ke Jia updated SPARK-30549: --- Summary: Fix the subquery metrics showing issue in UI When enable AQE (was: Fix the subquery metrics

[jira] [Updated] (SPARK-30549) Fix the subquery metrics showing issue in UI When enable AQE

[ https://issues.apache.org/jira/browse/SPARK-30549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ke Jia updated SPARK-30549: --- Description: After merged [https://github.com/apache/spark/pull/25316], the subquery metrics can not be

[jira] [Created] (SPARK-30549) Fix the subquery metrics showing issue in UI

Ke Jia created SPARK-30549: -- Summary: Fix the subquery metrics showing issue in UI Key: SPARK-30549 URL: https://issues.apache.org/jira/browse/SPARK-30549 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-30319) Adds a stricter version of as[T]

[ https://issues.apache.org/jira/browse/SPARK-30319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-30319: -- Affects Version/s: (was: 2.4.4) 3.0.0 > Adds a stricter version of

[jira] [Updated] (SPARK-30319) Adds a stricter version of as[T]

[ https://issues.apache.org/jira/browse/SPARK-30319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-30319: -- Fix Version/s: (was: 3.0.0) > Adds a stricter version of as[T] >

[jira] [Commented] (SPARK-30542) Two Spark structured streaming jobs cannot write to same base path

[ https://issues.apache.org/jira/browse/SPARK-30542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017820#comment-17017820 ] Sivakumar commented on SPARK-30542: --- Earlier with Spark Dstreams two jobs can have a same base path.

[jira] [Updated] (SPARK-30542) Two Spark structured streaming jobs cannot write to same base path

[ https://issues.apache.org/jira/browse/SPARK-30542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sivakumar updated SPARK-30542: -- Description: Hi All, Spark Structured Streaming doesn't allow two structured streaming jobs to write

[jira] [Resolved] (SPARK-30282) UnresolvedV2Relation should be resolved to temp view first

[ https://issues.apache.org/jira/browse/SPARK-30282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30282. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26921

[jira] [Assigned] (SPARK-30282) UnresolvedV2Relation should be resolved to temp view first

[ https://issues.apache.org/jira/browse/SPARK-30282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-30282: --- Assignee: Terry Kim > UnresolvedV2Relation should be resolved to temp view first >

[jira] [Created] (SPARK-30548) Cached blockInfo in BlockMatrix.scala is never released

Dong Wang created SPARK-30548: - Summary: Cached blockInfo in BlockMatrix.scala is never released Key: SPARK-30548 URL: https://issues.apache.org/jira/browse/SPARK-30548 Project: Spark Issue

[jira] [Commented] (SPARK-30530) CSV load followed by "is null" filter produces incorrect results

[ https://issues.apache.org/jira/browse/SPARK-30530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017806#comment-17017806 ] Maxim Gekk commented on SPARK-30530: [~jlowe] I prepared a fix for the issue. [~hyukjin.kwon]

[jira] [Created] (SPARK-30547) Add unstable annotation to the CalendarInterval class

Kent Yao created SPARK-30547: Summary: Add unstable annotation to the CalendarInterval class Key: SPARK-30547 URL: https://issues.apache.org/jira/browse/SPARK-30547 Project: Spark Issue Type:

[jira] [Updated] (SPARK-30544) Upgrade Genjavadoc to 0.15

[ https://issues.apache.org/jira/browse/SPARK-30544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-30544: --- Parent: SPARK-25075 Issue Type: Sub-task (was: Improvement) > Upgrade Genjavadoc

[jira] [Commented] (SPARK-30542) Two Spark structured streaming jobs cannot write to same base path

[ https://issues.apache.org/jira/browse/SPARK-30542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017791#comment-17017791 ] Jungtaek Lim commented on SPARK-30542: -- This is more likely a question rather than actual bug which

[jira] [Created] (SPARK-30546) Make interval type more future-proofing

Kent Yao created SPARK-30546: Summary: Make interval type more future-proofing Key: SPARK-30546 URL: https://issues.apache.org/jira/browse/SPARK-30546 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-30544) Upgrade Genjavadoc to 0.15

[ https://issues.apache.org/jira/browse/SPARK-30544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-30544: --- Affects Version/s: (was: 3.1.0) 3.0.0 > Upgrade Genjavadoc to

[jira] [Updated] (SPARK-30543) RandomForest add Param bootstrap to control sampling method

[ https://issues.apache.org/jira/browse/SPARK-30543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-30543: - Issue Type: Improvement (was: Bug) > RandomForest add Param bootstrap to control sampling

[jira] [Created] (SPARK-30545) Impl Extremely Randomized Trees

zhengruifeng created SPARK-30545: Summary: Impl Extremely Randomized Trees Key: SPARK-30545 URL: https://issues.apache.org/jira/browse/SPARK-30545 Project: Spark Issue Type: New Feature

[jira] [Assigned] (SPARK-30545) Impl Extremely Randomized Trees

[ https://issues.apache.org/jira/browse/SPARK-30545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng reassigned SPARK-30545: Assignee: zhengruifeng > Impl Extremely Randomized Trees >

[jira] [Created] (SPARK-30544) Upgrade Genjavadoc to 0.15

Kousuke Saruta created SPARK-30544: -- Summary: Upgrade Genjavadoc to 0.15 Key: SPARK-30544 URL: https://issues.apache.org/jira/browse/SPARK-30544 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-30462) Structured Streaming _spark_metadata fills up Spark Driver memory when having lots of objects

[ https://issues.apache.org/jira/browse/SPARK-30462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017782#comment-17017782 ] Sivakumar commented on SPARK-30462: --- Hi All, I have two structured streaming jobs which should write