smengcl commented on code in PR #6413:
URL: https://github.com/apache/ozone/pull/6413#discussion_r1560107838
##########
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BufferPool.java:
##########
@@ -79,76 +83,88 @@ ChunkBuffer getCurrentBuffer() {
* less than the capacity to be allocated, just allocate a buffer of size
* chunk size.
*/
- public ChunkBuffer allocateBuffer(int increment) {
+ public synchronized ChunkBuffer allocateBuffer(int increment) {
final int nextBufferIndex = currentBufferIndex + 1;
Preconditions.assertTrue(nextBufferIndex < capacity, () ->
"next index: " + nextBufferIndex + " >= capacity: " + capacity);
currentBufferIndex = nextBufferIndex;
+ LOG.warn("!! allocateBuffer(increment = {}): " +
+ "capacity = {}, currentBufferIndex = {}, nextBufferIndex = {},
bufferList.size() = {}",
+ increment,
+ capacity, currentBufferIndex, nextBufferIndex, bufferList.size());
+
if (currentBufferIndex < bufferList.size()) {
return getBuffer(currentBufferIndex);
} else {
final ChunkBuffer newBuffer = ChunkBuffer.allocate(bufferSize,
increment);
+ LOG.warn("!! allocateBuffer(): ChunkBuffer allocated: {}", newBuffer);
bufferList.add(newBuffer);
return newBuffer;
}
}
- void releaseBuffer(ChunkBuffer chunkBuffer) {
+ synchronized void releaseBuffer(ChunkBuffer chunkBuffer) {
+ LOG.warn("!! releaseBuffer(chunkBuffer = {}):" +
+ "currentBufferIndex = {}, bufferList.indexOf(chunkBuffer) = {}",
+ chunkBuffer,
+ currentBufferIndex, bufferList.indexOf(chunkBuffer));
+
Preconditions.assertTrue(!bufferList.isEmpty(), "empty buffer list");
- Preconditions.assertSame(bufferList.get(0), chunkBuffer,
- "only the first buffer can be released");
+// Preconditions.assertSame(bufferList.get(0), chunkBuffer,
+// "only the first buffer can be released");
Preconditions.assertTrue(currentBufferIndex >= 0,
() -> "current buffer: " + currentBufferIndex);
// always remove from head of the list and append at last
- final ChunkBuffer buffer = bufferList.remove(0);
- buffer.clear();
- bufferList.add(buffer);
- currentBufferIndex--;
+ final boolean res = bufferList.remove(chunkBuffer);
Review Comment:
This change (of attempting to remove buffer out-of-order) is not working as
we expected. This is likely the culprit of buffer leak (aka Issue 2 mentioned
above).
I did an experiment of triggering `bufferList.remove(chunkBuffer)` manually
and proved this to be an issue.
Example: The buffer to remove is 14606. There are buffer 14615 and buffer
14606 in the list:
<img width="766" alt="Screenshot 2024-04-10 at 2 58 55 PM"
src="https://github.com/apache/ozone/assets/50227127/769e0b71-d6a7-4018-8428-965024cad1d9";>
When I trigger `bufferList.remove(chunkBuffer)`, buffer 14615 got removed
(!) but not buffer 14606:
<img width="778" alt="Screenshot 2024-04-10 at 2 59 19 PM"
src="https://github.com/apache/ozone/assets/50227127/fb19c0bf-1e00-4cc5-b076-f7c793d00dff";>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
