On Wed, Aug 22, 2018 at 4:57 PM, Jamo Luhrsen <jluhr...@gmail.com> wrote:
> I am looking for some help/ideas. I am running three containers
> on my laptop so I can test some cluster bugs locally. I am fine
> with the netty based akka remoting (our default), but I've been
> asked to reproduce a bug with the artery remoting.
>
> Every time I start my controllers with artery, they all die with:
>
> ERROR | opendaylight-cluster-data-akka.actor.default-dispatcher-56 |
> ActorSystemImpl | 41 -
> com.typesafe.akka.slf4j - 2.5.11 | Uncaught error from thread
> [opendaylight-cluster-data-akka.remote.default-remote-dispatcher-7]: Di
> rect buffer memory, shutting down JVM since 'akka.jvm-exit-on-fatal-error'
> is enabled for ActorSystem[opendaylight-cluster-data]
> java.lang.OutOfMemoryError: Direct buffer memory
> at java.nio.Bits.reserveMemory(Bits.java:695) ~[?:?]
> at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
> ~[?:?]
> at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311) ~[?:?]
> at
> akka.remote.artery.EnvelopeBufferPool.acquire(EnvelopeBufferPool.scala:34)
> ~[40:com.typesafe.akka.remote:2.5.11]
> at akka.remote.artery.Encoder$$anon$2.onPush(Codecs.scala:93)
> ~[40:com.typesafe.akka.remote:2.5.11]
>
>
> I'm stuck in the mud on this one so far. I'm trying to tweak the
> shared memory setting in the docker run command (to no avail).
>
> The container stats are nowhere near any kind of limit.
>
> It's not happening with netty based.
>
> I've tried both udp and tcp protocols with the same OOM death.
>
> appreciate any ideas or pointers. I'm literally just throwing mud against
> the wall at this point hoping something will stick.
>
> TomP tells me it works on his laptop, but he's not using docker/containers,
> so that must be a clue.
>
The problem is this in configuration/factory/akka.conf:
artery {
advanced {
maximum-frame-size = 1 GiB
maximum-large-frame-size = 1 GiB
}
}
I had set these originally a while ago when we were first looking at artery
before I realized it was using direct memory. I forgot to remove them. I'll
submit a patch to do that - in the mean time you can remove it locally.
>
> Thanks,
> JamO
>
_______________________________________________
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev
