fayce66 opened a new issue #9317:
URL: https://github.com/apache/pulsar/issues/9317
C++ consumer configured for topic compaction still receives all messages in
the log instead of the last one for each topic key.
1. The c++ producer:
```
std::string some_string;
std::string some_topic_key;
Client client(lookup_url_);
Producer producer;
ProducerConfiguration configuration;
configuration.setCompressionType(compression_type_); // CompressionNone
or CompressionZSTD
auto result = client.createProducer(topic_, configuration,producer);
if (result != ResultOk) {
// print error
return -1;
}
auto msg = MessageBuilder()
.setContent(some_string.data(), some_string.size())
.setPartitionKey(some_topic_key)
.build();
auto res = producer.send(msg);
```
2. The c++ consumer:
```
Client client(lookup_url_);
Consumer consumer;
ConsumerConfiguration configuration;
configuration.setConsumerType(consumer_type_); // ConsumerExclusive or
ConsumerFailover
if (consumer_type_ == pulsar::ConsumerShared) {
// cannot have topic compaction with shared subscription ?
configuration.setReadCompacted(false);
} else {
// set topic compaction
configuration.setReadCompacted(true);
}
auto result = client.subscribe(topic_, subscription_name_,
configuration, consumer);
if (result != ResultOk) {
// print error
return -1;
}
Message msg;
while (true) {
consumer.receive(msg);
consumer.acknowledge(msg);
}
```
3. set manual topic compaction with pulsar-admin:
`$ bin/pulsar-admin topics compact
"persistent://marianas/alphatrader/wing-calibration"
`
desktop: ubuntu 20.04 client (producer & consumer), pulsar-daemon running on
centos 6.10
I checked the messages received by the pulsar-client and the topic keys are
properly set:
----- got message -----
key:[fek.wing.ks102], properties:[], content:{"some string"}
----- got message -----
key:[fek.wing.ks103], properties:[], content:{"some string"}
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
