[
https://issues.apache.org/jira/browse/HDDS-11009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17855580#comment-17855580
]
Tsz-wo Sze commented on HDDS-11009:
-----------------------------------
{code}
2024-06-13 06:55:05,217 [om1-OMStateMachineApplyTransactionThread - 0] DEBUG
db.CodecBuffer (CodecBuffer.java:setCapacity(329)) - setCapacity: 4096 ->
15067, max=2147483647
{code}
Also, there were a lot of "CodecBuffer setCapacity" as shown above in the log.
Not sure if it is related. Let's print more information:
{code}
diff --git
a/hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/utils/db/CodecBuffer.java
b/hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/utils/db/CodecBuffer.java
index 1ac293b301..d4b418cd2b 100644
---
a/hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/utils/db/CodecBuffer.java
+++
b/hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/utils/db/CodecBuffer.java
@@ -37,11 +37,14 @@
import java.io.InputStream;
import java.io.OutputStream;
import java.nio.ByteBuffer;
+import java.util.Map;
import java.util.Objects;
import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.BiFunction;
import java.util.function.IntFunction;
+import java.util.function.Supplier;
import java.util.function.ToIntFunction;
import static org.apache.hadoop.hdds.HddsUtils.formatStackTrace;
@@ -53,6 +56,14 @@
*/
public class CodecBuffer implements UncheckedAutoCloseable {
public static final Logger LOG = LoggerFactory.getLogger(CodecBuffer.class);
+ private static final Supplier<Map<String, String>> TRACE_MAP =
MemoizedSupplier.valueOf(ConcurrentHashMap::new);
+
+ private static String getFirstTrace(Throwable t) {
+ final String trace =
org.apache.ratis.util.StringUtils.stringifyException(t);
+ final String returned = TRACE_MAP.get().putIfAbsent(trace, trace);
+ // print only if the returned is the same object as trace, i.e. ==
+ return trace == returned ? trace : "";
+ }
/** To create {@link CodecBuffer} instances. */
private static class Factory {
@@ -326,8 +337,11 @@ public boolean setCapacity(int newCapacity) {
throw new IllegalArgumentException(
"newCapacity = " + newCapacity + " < 0");
}
- LOG.debug("setCapacity: {} -> {}, max={}",
- buf.capacity(), newCapacity, buf.maxCapacity());
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("setCapacity({}): {} -> {}, max={} {}",
+ isDirect() ? "direct" : "heap", buf.capacity(), newCapacity,
buf.maxCapacity(),
+ getFirstTrace(new Throwable("TRACE")));
+ }
if (newCapacity <= buf.maxCapacity()) {
final ByteBuf returned = buf.capacity(newCapacity);
Preconditions.assertSame(buf, returned, "buf");
{code}
> OutOfMemoryError in TestRandomKeyGenerator with Ratis 3.1.0 rc1
> ---------------------------------------------------------------
>
> Key: HDDS-11009
> URL: https://issues.apache.org/jira/browse/HDDS-11009
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Attila Doroszlai
> Priority: Major
>
> {{TestRandomKeyGenerator}} is failing with {{OutOfMemoryError}} when running
> Ozone CI with Ratis 3.1.0 rc1 ([runs
> 1-4|https://github.com/adoroszlai/ozone/actions/runs/9479020635], [run
> 5|https://github.com/adoroszlai/ozone/actions/runs/9494919684/job/26166730012]).
> However, it
> [passed|https://github.com/adoroszlai/ozone/actions/runs/9485698062] 100x in
> {{flaky-test-check}} building {{ratis-3.1.0-rc1}} from source.
> Also passes locally.
> CC [~szetszwo]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@ozone.apache.org
For additional commands, e-mail: issues-h...@ozone.apache.org
