Claus Ibsen created CAMEL-26610:
-----------------------------------
Summary: camel-test-dst - more fake worlds for the consumers of
other components (kafka, jms, file/ftp, cloud queues, reconnecting consumers)
Key: CAMEL-26610
URL: https://issues.apache.org/jira/browse/CAMEL-26610
Project: Camel
Issue Type: Improvement
Components: camel-test
Reporter: Claus Ibsen
Fix For: 4.24.0
camel-test-dst (CAMEL-25738) runs the real consumer of a component under a
seeded virtual-time scheduler against a fake of the external system. Five
components have a fake world today, listed in the catalog ({{dstWorlds}},
CAMEL-26438): aws2-sqs, azure-eventhubs, kafka-share, pulsar and slack. They
found real bugs, for example CAMEL-25737 (the SQS visibility extender is always
late with visibilityTimeout=1).
This ticket is for more worlds, for the consumers where timing bugs happen:
checkpoint, commit, acknowledge, visibility or lock renewal, reconnect, suspend
and resume, stop while a poll is running. Each world gets its own JIRA (one per
component) when work starts, linked here.
h3. Rules for each world
* Follow "Writing a fake world" in
{{components/camel-test/camel-test-dst/src/main/docs/test-dst.adoc}}; the SQS
world is the worked example.
* The fake follows the documentation of the external system, cited in the code,
not what the Camel consumer expects.
* *Self-check against an old fix:* the world must reproduce at least one of the
fixed bugs listed below on the code from before its fix, and pass on main. That
shows the world can find that class of bug.
* Mark it with {{@DstWorld}} so the catalog and the docs list it.
* Do not report a failing seed as a bug before the triage step of the guide.
h3. Suggested order
The order weighs how many users the component has, how much timing logic its
consumer has, the timing bugs fixed in it this year, and how easy it is to fake
the client.
*Tier 1: widely used, complex consumer timing*
# *camel-kafka*: commit strategies (sync, async, manual), breakOnFirstError,
pause and resume, rebalance, batching. Fixed: CAMEL-25756 (a failed poll
dropped the records of an incomplete batch), CAMEL-25753 (stop waited for the
current poll), CAMEL-25313 (a pause or resume request was lost while
suspended). Seam: kafka-clients ships {{MockConsumer}}, which may serve as the
base of the fake; the kafka-share world shows the approach.
# *camel-jms / camel-sjms*: the InOut reply manager (request timeout against a
late reply against a failed send), transacted and CLIENT_ACKNOWLEDGE consumers,
stop and start of the listener. Fixed: CAMEL-25095 and CAMEL-25122 (an InOut
exchange completed twice), CAMEL-26536 (asyncStopListener lost the listener
container). Harder seam (the Spring listener container has its own threads),
but the double completion bugs make it worth it.
# *camel-file / camel-ftp / camel-sftp*: read locks, the in-progress and
idempotent repositories, moveFailed, done files, reconnect after a failed poll.
Fixed: CAMEL-25856 (the lock of a file name released while in use), CAMEL-25728
(done files), CAMEL-25915 and CAMEL-25896 (ftp reconnect and noop). Seam:
{{GenericFileOperations}} / {{RemoteFileOperations}}.
*Tier 2: cloud queues with ack deadlines and lock renewal (the same family as
the SQS world)*
# *camel-azure-servicebus*: message lock renewal. Fixed: CAMEL-25701 (the
renewal came too late, the same class as CAMEL-25737).
# *camel-google-pubsub*: ack deadline extension, nack at max delivery attempts,
stop. Fixed: CAMEL-25712, CAMEL-25790.
# *camel-aws2-kinesis* and the *camel-aws2-s3* consumer: shard checkpoints and
resume, the in-progress repository. Fixed: CAMEL-25942, CAMEL-25558,
CAMEL-25521.
*Tier 3: connection and subscription consumers (reconnect, resubscribe)*
# *camel-milo*: the most timing fixes this year, about reconnects and
subscriptions. Fixed: CAMEL-25691, CAMEL-25812, CAMEL-25873, CAMEL-26570.
# *camel-paho / camel-paho-mqtt5 / camel-hivemq*: reconnect, resubscribe,
manual acks. Fixed: CAMEL-24523, CAMEL-25944, CAMEL-25793.
# *camel-nats* (JetStream) and *camel-rabbitmq / camel-spring-rabbitmq*: acks,
stop while subscribing. Fixed: CAMEL-26462, CAMEL-25847.
# *camel-netty* client mode and *camel-vertx-websocket*: reconnect without
blocking the event loop, consumers of a shared host. Fixed: CAMEL-26574,
CAMEL-26573.
*Tier 4: leader election and change streams*
# *camel-zookeeper-master* and the cluster services: leadership lost while the
consumer starts, retry of a failed start. Fixed: CAMEL-26540, CAMEL-25315;
open: CAMEL-26577 (RebalancingCamelClusterService).
# *camel-mongodb* change streams and *camel-mongodb-gridfs*: the consumer goes
on after its collection is dropped or a poll fails. Fixed: CAMEL-25946,
CAMEL-25711, CAMEL-25607.
camel-seda (11 timing fixes this year) has no external system, so it needs
deterministic tests in camel-core rather than a fake world.
The order is a suggestion; a contributor who knows one of these systems well
can pick it first.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
