michalcukierman commented on issue #21104: URL: https://github.com/apache/pulsar/issues/21104#issuecomment-1743609860
It looks like I was able to reproduce the issue in the first run today. - I've created SmallRye project (one producer to produce 1 mln messages, one consumer to read it using shared subscription) - The cluster is created on GCP using Pulsar operator from SN (great tool!) The code is here: https://github.com/michalcukierman/pulsar-21104 In general it's very much like in the bug description. Produce 1 mln messages of 30kb: ```java @Outgoing("requests-out") public Multi<String> produce() { return Multi.createBy().repeating() .uni(() -> Uni .createFrom() .item(() -> RandomStringUtils.randomAlphabetic(30_000)) .onItem() .invoke(() -> System.out.println("+ Produced: " + outCount.incrementAndGet())) ) .atMost(1_000_000); } ``` Read it using shared client and write to another topic: ```java @ApplicationScoped public class Processor { private final AtomicLong inCount = new AtomicLong(0); @Incoming("requests-in") @Outgoing("dump-out") @Blocking PulsarOutgoingMessage<String> process(PulsarIncomingMessage<String> in) { System.out.println(" - Processed: " + inCount.incrementAndGet()); return PulsarOutgoingMessage.from(in); } } ``` The settings of the client are: ``` pulsar.client.serviceUrl=pulsar://brokers-broker:6650 mp.messaging.incoming.requests-in.subscriptionType=Shared mp.messaging.incoming.requests-in.numIoThreads=4 mp.messaging.incoming.requests-in.subscriptionName=request-shared-subscription mp.messaging.incoming.requests-in.ackTimeoutMillis=5000 mp.messaging.incoming.requests-in.subscriptionInitialPosition=Earliest mp.messaging.incoming.requests-in.receiverQueueSize=8 mp.messaging.incoming.requests-in.topic=persistent://public/default/requests_4 mp.messaging.incoming.requests-in.connector=smallrye-pulsar mp.messaging.outgoing.dump-out.topic=persistent://public/default/dump mp.messaging.outgoing.dump-out.connector=smallrye-pulsar mp.messaging.outgoing.dump-out.blockIfQueueFull=true mp.messaging.outgoing.dump-out.maxPendingMessages=8 mp.messaging.outgoing.dump-out.maxPendingMessagesAcrossPartitions=12 ``` The retention of the topic `requests` is set using Pulsar Admin in Java to -1 -1. During two runs the consumer get stucked: <img width="1013" alt="Screenshot 2023-10-02 at 21 12 42" src="https://github.com/apache/pulsar/assets/4356553/b31bb39c-1491-4852-8eb6-a975155f31b2";> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
