[akka-user] Re: OutOfMemoryError due to too much logging
Hi Adam, I think what you are describing is covered in the logging docs: http://doc.akka.io/docs/akka/2.4.1/scala/logging.html#Loggers and http://doc.akka.io/docs/akka/2.4.1/java/logging.html#Loggers It could probably be more clear though, but the system behaving bad with high logging load and configuring an async appender and dropping DEBUG and INFO messages on high load is mentioned. -- Johan Andrén Typesafe - Reactive apps on the JVM Twitter: @apnylle On Sunday, December 13, 2015 at 5:15:36 PM UTC+1, Adam wrote: > > Hi, > > I sometimes see this issue where I want to switch logs to DEBUG, but due > to the large number of messages the JVM will crash on OutOfMemory which > turns out to be due to the accumulation of logging events. > Most logging frameworks have the ability to drop messages beyond a certain > number and so do Akka unbounded mailboxes. > > Is there a way to achieve this with Akka logging as well? > The behavior of the non blocking unbounded mailbox is what I have in mind > (logging frameworks usually add a warning of some kind telling you that > messages are being dropped, but I can manage without this for now). > > Alternatively, I could just using my logger directly. > It's asynchronous anyway. The only issue with that is that I find the > extra MDC data of actor path very useful when looking at piles of DEBUG > logs. > It makes zeroing in on a single transaction very simple. > -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: OutOfMemoryError due to too much logging
Hi, I've read it in the past, but was hoping it's not accurate (I'm already using an asynchronous logger as the backend). It basically says that no - there is no way to achieve what I want with Akka Logging, because while an asynchronous backend may help, the messages still go through some sort of unbounded queue. So in the case of a fast producer (lots of logging requests) and a slow consumer (logging backend reading from this queue), you will still get an OutOfMemoryError. OK, so I guess I'll just use those loggers directly and I'll have to populate the MDC on my own. Probably not that much work anyway. Thanks! On Monday, December 14, 2015 at 10:53:13 AM UTC+2, Johan Andrén wrote: > > Hi Adam, > > I think what you are describing is covered in the logging docs: > http://doc.akka.io/docs/akka/2.4.1/scala/logging.html#Loggers and > http://doc.akka.io/docs/akka/2.4.1/java/logging.html#Loggers > > It could probably be more clear though, but the system behaving bad with > high logging load and configuring an async appender and dropping DEBUG and > INFO messages on high load is mentioned. > > -- > Johan Andrén > Typesafe - Reactive apps on the JVM > Twitter: @apnylle > > On Sunday, December 13, 2015 at 5:15:36 PM UTC+1, Adam wrote: >> >> Hi, >> >> I sometimes see this issue where I want to switch logs to DEBUG, but due >> to the large number of messages the JVM will crash on OutOfMemory which >> turns out to be due to the accumulation of logging events. >> Most logging frameworks have the ability to drop messages beyond a >> certain number and so do Akka unbounded mailboxes. >> >> Is there a way to achieve this with Akka logging as well? >> The behavior of the non blocking unbounded mailbox is what I have in mind >> (logging frameworks usually add a warning of some kind telling you that >> messages are being dropped, but I can manage without this for now). >> >> Alternatively, I could just using my logger directly. >> It's asynchronous anyway. The only issue with that is that I find the >> extra MDC data of actor path very useful when looking at piles of DEBUG >> logs. >> It makes zeroing in on a single transaction very simple. >> > -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: OutOfMemoryError due to too much logging
Hello! I saw the same problem when default dispatcher was overloaded by a lot of small tasks that was created via map/flatMap/etc operations on Future's. I fixed my problem by moving away heavy batch processing and operations on futures to dedicated dispatcher. Maxim воскресенье, 13 декабря 2015 г., 19:15:36 UTC+3 пользователь Adam написал: > > Hi, > > I sometimes see this issue where I want to switch logs to DEBUG, but due > to the large number of messages the JVM will crash on OutOfMemory which > turns out to be due to the accumulation of logging events. > Most logging frameworks have the ability to drop messages beyond a certain > number and so do Akka unbounded mailboxes. > > Is there a way to achieve this with Akka logging as well? > The behavior of the non blocking unbounded mailbox is what I have in mind > (logging frameworks usually add a warning of some kind telling you that > messages are being dropped, but I can manage without this for now). > > Alternatively, I could just using my logger directly. > It's asynchronous anyway. The only issue with that is that I find the > extra MDC data of actor path very useful when looking at piles of DEBUG > logs. > It makes zeroing in on a single transaction very simple. > -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Re: OutOfMemoryError due to too much logging
That sounds related, but the problem in the case of logging is that (as far as I know) it runs on the default dispatcher and cannot be reconfigured. And anyway, I'm thinking of a case where regardless of dispatcher, there's just too many messages and we cannot possibly manage to write them all. On Mon, Dec 14, 2015 at 11:31 AM, Maxim Valyanskiy wrote: > Hello! > > I saw the same problem when default dispatcher was overloaded by a lot of > small tasks that was created via map/flatMap/etc operations on Future's. I > fixed my problem by moving away heavy batch processing and operations on > futures to dedicated dispatcher. > > Maxim > > воскресенье, 13 декабря 2015 г., 19:15:36 UTC+3 пользователь Adam написал: > >> Hi, >> >> I sometimes see this issue where I want to switch logs to DEBUG, but due >> to the large number of messages the JVM will crash on OutOfMemory which >> turns out to be due to the accumulation of logging events. >> Most logging frameworks have the ability to drop messages beyond a >> certain number and so do Akka unbounded mailboxes. >> >> Is there a way to achieve this with Akka logging as well? >> The behavior of the non blocking unbounded mailbox is what I have in mind >> (logging frameworks usually add a warning of some kind telling you that >> messages are being dropped, but I can manage without this for now). >> >> Alternatively, I could just using my logger directly. >> It's asynchronous anyway. The only issue with that is that I find the >> extra MDC data of actor path very useful when looking at piles of DEBUG >> logs. >> It makes zeroing in on a single transaction very simple. >> > -- > >> Read the docs: http://akka.io/docs/ > >> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to a topic in the > Google Groups "Akka User List" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/akka-user/Ct9noZyBhtU/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > akka-user+unsubscr...@googlegroups.com. > To post to this group, send email to akka-user@googlegroups.com. > Visit this group at https://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
