[
https://issues.apache.org/jira/browse/BEAM-12523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17549482#comment-17549482
]
Danny McCormick commented on BEAM-12523:
----------------------------------------
This issue has been migrated to https://github.com/apache/beam/issues/21075
> Number of connection issue
> --------------------------
>
> Key: BEAM-12523
> URL: https://issues.apache.org/jira/browse/BEAM-12523
> Project: Beam
> Issue Type: Bug
> Components: io-java-jms
> Affects Versions: 2.30.0
> Reporter: Vincent BALLADA
> Priority: P3
>
> The maximum number of connections using JmsIO.Read seems to grow in an
> uncontrolled way, causing new connections to be rejected by the broker if the
> number exceeds the configured quota on broker side.
> The number of connections is related to the number of workers allocated by
> the runner, and the number of threads configured per worker, at least at the
> beginning.
> Our assumption is:
> JmsCheckPointMark is mutable (private Instant oldestMessageTimestamp – this
> field is updated for each message received). What we think it’s happening,
> with direct runner at least, the runner is unable to retrieve existing JMSIO
> readers (JmsCheckPointMark is used as a key - as part of splitable pardo
> restriction object - for retrieving active JMSIO Readers, and because it’s
> mutated for each received message, the runner will create a new reader for
> each new received message – that’s why the connections number spins out of
> control at least in DirectRunner). This is the reason that target parallelism
> parameter helps on controlling the ‘initial’ number of active connections,
> but later on we have issues.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
