[
https://issues.apache.org/jira/browse/SPARK-37391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Guinther updated SPARK-37391:
-----------------------------------
Description:
The fix for https://issues.apache.org/jira/browse/SPARK-32001 (
[https://github.com/apache/spark/pull/29024/files#diff-345beef18081272d77d91eeca2d9b5534ff6e642245352f40f4e9c9b8922b085R58]
) does not seem to have consider the reality that some apps may rely on being
able to establish many JDBC connections simultaneously for performance reasons.
The fix forces concurrency to 1 when establishing database connections and that
strikes me as a *significant* user impacting change and a *significant*
bottleneck.
Can anyone propose a workaround for this? I have an app that makes connections
to thousands of databases and I can't upgrade to any version >3.1.x because of
this significant bottleneck.
Thanks in advance for your help!
was:
The fix for SPARK-34497 (
[https://github.com/apache/spark/pull/29024/files#diff-345beef18081272d77d91eeca2d9b5534ff6e642245352f40f4e9c9b8922b085R58|https://github.com/apache/spark/pull/29024/files#diff-345beef18081272d77d91eeca2d9b5534ff6e642245352f40f4e9c9b8922b085R58]
) does not seem to have consider the reality that some apps may rely on being
able to establish many JDBC connections simultaneously for performance reasons.
The fix forces concurrency to 1 when establishing database connections and that
strikes me as a *significant* user impacting change and a *significant*
bottleneck.
Can anyone propose a workaround for this? I have an app that makes connections
to thousands of databases and I can't upgrade to any version >3.1.x because of
this significant bottleneck.
Thanks in advance for your help!
> SIGNIFICANT bottleneck introduced by fix for SPARK-34497
> --------------------------------------------------------
>
> Key: SPARK-37391
> URL: https://issues.apache.org/jira/browse/SPARK-37391
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.1.0, 3.1.1, 3.1.2, 3.2.0
> Environment: N/A
> Reporter: Danny Guinther
> Priority: Major
>
> The fix for https://issues.apache.org/jira/browse/SPARK-32001 (
> [https://github.com/apache/spark/pull/29024/files#diff-345beef18081272d77d91eeca2d9b5534ff6e642245352f40f4e9c9b8922b085R58]
> ) does not seem to have consider the reality that some apps may rely on
> being able to establish many JDBC connections simultaneously for performance
> reasons.
> The fix forces concurrency to 1 when establishing database connections and
> that strikes me as a *significant* user impacting change and a *significant*
> bottleneck.
> Can anyone propose a workaround for this? I have an app that makes
> connections to thousands of databases and I can't upgrade to any version
> >3.1.x because of this significant bottleneck.
>
> Thanks in advance for your help!
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
