RobertIndie opened a new pull request, #110:
URL: https://github.com/apache/pulsar-client-cpp/pull/110
<!--
### Contribution Checklist
- PR title format should be *[type][component] summary*. For details, see
*[Guideline - Pulsar PR Naming
Convention](https://docs.google.com/document/d/1d8Pw6ZbWk-_pCKdOmdvx9rnhPiyuxwq60_TrD68d7BA/edit#heading=h.trs9rsex3xom)*.
- Fill out the template below to describe the changes contributed by the
pull request. That will give reviewers the context they need to do the review.
- Each pull request should address only one issue, not mix up code from
multiple issues.
- Each commit in the pull request has a meaningful commit message
- Once all items of the checklist are addressed, remove the above text and
this checklist, leaving only the filled out template below.
-->
<!-- Either this PR fixes an issue, -->
Fixes #104
### Motivation
Currently, the consumer ack the last message when the chunked messages
exceed maxPendingChunkMessages. This is wrong behavior. This may lead to
unexpected data loss.
This PR also fixes serval issues related to maxPendingChunkedMessages:
https://github.com/apache/pulsar-client-cpp/blob/1f7fdb86c409c1486d160528137f10ce07dcf3b2/lib/ConsumerImpl.cc#L387-L407
In the current logic, there are two `putIfAbsent` operations here, and they
are confusing. If a new chunk message is received, it will be added to the
chunkedMessageCache. But if the size of the cache reaches the
maxPendingChunkedMessages, it will remove at least 1 ctx from the cache due to
`chunkedMessageCache_.size() - maxPendingChunkedMessage_ + 1`. But the message
is then put into the cache again. This can lead to unnecessary ctx buffer
memory allocations.
Here are some key point of this issue:
<img width="961" alt="image"
src="https://user-images.githubusercontent.com/16974619/200777662-59c2e262-fb68-4f03-b1b0-8a312cb1ac58.png";>
### Modifications
* Fix consumer acked the wrong message when pending chunked messages exceed
maxPendingChunkMessages
* Fix wrong behavior when remove the ctx from the chunkedMessageCache.
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
*(Please pick either of the following options)*
This change is a trivial rework / code cleanup without any test coverage.
*(or)*
This change is already covered by existing tests, such as *(please describe
tests)*.
*(or)*
This change added tests and can be verified as follows:
*(example:)*
- *Added integration tests for end-to-end deployment with large payloads
(10MB)*
- *Extended integration test for recovery after broker failure*
### Documentation
<!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
- [ ] `doc-required`
(Your PR needs to update docs and you will update later)
- [x] `doc-not-needed`
(Please explain why)
- [ ] `doc`
(Your PR contains doc changes)
- [ ] `doc-complete`
(Docs have been already added)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
![https://user-images.githubusercontent.com/16974619/200777662-59c2e262-fb68-4f03-b1b0-8a312cb1ac58.png"](https://user-images.githubusercontent.com/16974619/200777662-59c2e262-fb68-4f03-b1b0-8a312cb1ac58.png"){kind=link}