Claus Ibsen created CAMEL-27192:
-----------------------------------
Summary: camel-google-pubsub - publisher cache ignores ordering,
no resumePublish
Key: CAMEL-27192
URL: https://issues.apache.org/jira/browse/CAMEL-27192
Project: Camel
Issue Type: Bug
Components: camel-google-pubsub
Reporter: Claus Ibsen
Two problems with message ordering in the producer, found by reading the code
while adding {{messageOrderingEnabled}} to the {{google-pubsub-sink}} Kamelet.
h3. 1. The publisher cache is keyed by topic only
{{GooglePubsubComponent.getPublisher}} caches one {{Publisher}} per topic name:
{code:java}
return cachedPublishers.get(topicName, () -> buildPublisher(topicName,
googlePubsubEndpoint));
{code}
but {{buildPublisher}} configures the publisher from the endpoint:
{{messageOrderingEnabled}}, {{pubsubEndpoint}} and {{retry}}. A second endpoint
publishing to the same topic with different settings gets the publisher built
for the first one. For example, with
{code}
to("google-pubsub:my-project:orders") // used
first
to("google-pubsub:my-project:orders?messageOrderingEnabled=true&pubsubEndpoint=europe-west1-pubsub.googleapis.com:443")
{code}
the second endpoint publishes without ordering and to the global endpoint, and
its messages that carry {{CamelGooglePubsubOrderingKey}} fail with {{Cannot
publish a message with an ordering key when message ordering is not enabled}}
(the error seen when ordering is off). Which endpoint wins depends on which one
sends first after the cache entry expires ({{publisherCacheTimeout}}).
Suggested fix: key the cache by the topic plus the endpoint settings that
{{buildPublisher}} uses (or cache per endpoint).
h3. 2. No resumePublish after a failed ordered publish
With ordering enabled, when a publish with an ordering key fails, the Google
{{Publisher}} pauses that key: every later {{publish}} with the same key fails
at once until {{Publisher.resumePublish(orderingKey)}} is called.
{{GooglePubsubProducer.send}} calls {{publisher.publish(message)}} and
{{messageIdFuture.get()}} but never {{resumePublish}} ({{resumePublish}} is not
used anywhere in the component). One transient failure therefore blocks that
ordering key for good: the error handler's redeliveries and all later messages
with that key fail, until the cached publisher expires or the application
restarts.
Suggested fix: in {{send}}, when {{messageIdFuture.get()}} throws and the
message has an ordering key, call {{publisher.resumePublish(orderingKey)}}
before rethrowing, so that a redelivery or the next message can be published.
Document that ordering after a failure then depends on the route's error
handling (e.g. redelivery before continuing).
h3. Where
*
{{components/camel-google/camel-google-pubsub/src/main/java/org/apache/camel/component/google/pubsub/GooglePubsubComponent.java}}:
{{getPublisher}}, {{buildPublisher}}.
*
{{components/camel-google/camel-google-pubsub/src/main/java/org/apache/camel/component/google/pubsub/GooglePubsubProducer.java}}:
{{send}}.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)