Hi, I just created a patch. Please see https://github.com/apache/incubator-zeppelin/pull/762
Thanks, moon On Mon, Feb 29, 2016 at 10:01 PM Skanda <skanda.ganapa...@gmail.com> wrote: > Hi > > Im also hitting this issue of OOM for couple of notebooks with about 6 to > 7 paragraphs running Hive queries and caching 1000 records of output. > > Regards, > Skanda > > On Mon, Feb 29, 2016 at 10:42 PM, moon soo Lee <m...@apache.org> wrote: > >> Thanks for creating an issue. >> Let me look into it more. >> >> Best, >> moon >> >> On Sun, Feb 28, 2016 at 10:14 PM Dafeng Wang <dafe...@microsoft.com> >> wrote: >> >>> Thanks Moon for your effort to repro this bug, I had create a Jira >>> https://issues.apache.org/jira/browse/ZEPPELIN-706 for it, please let >>> me know if you need anything else. >>> >>> >>> >>> Regards, >>> >>> Dafeng >>> >>> >>> >>> *From:* moon soo Lee [mailto:m...@apache.org] >>> *Sent:* Saturday, February 27, 2016 2:25 AM >>> >>> >>> *To:* users@zeppelin.incubator.apache.org >>> *Subject:* Re: OOM error when run all paragraphs >>> >>> >>> >>> Thanks for sharing your use case and the memory usage table. >>> I could able to reproduce the problem. Do you mind create an issue for >>> it on our jira? >>> I might have sometime next week to dig into this problem. >>> >>> Thanks, >>> moon >>> >>> >>> >>> On Thu, Feb 25, 2016 at 5:49 PM Dafeng Wang <dafe...@microsoft.com> >>> wrote: >>> >>> Hi Moon, >>> >>> >>> >>> Thanks for your reply, as for my case: my zeppelin server only have 1 >>> notebook with almost all queries are sparkSql query(30 paragraphs), the >>> result limitation I set is 10000, all query reach the limitation of 10000. >>> >>> The strange thing is: run them one by one won’t cause the OOM error, if >>> I click on run notebooks, it will OOM quickly. >>> >>> >>> >>> I check on the memory usage, the fastest increasing part is [B and [C, >>> however, it just tells me we do have memory increase, doesn’t hint any >>> other things, I put the table below for reference, the last one is the >>> latest one, you can see only *scala.reflect.io.VirtualFile >>> <http://localhost:7000/refsByType/0x616022bc8> *got increased when I >>> run it again and again, no obvious clue of memory leak. >>> >>> >>> >>> *scala.reflect.io.FileZipArchive$FileEntry$1 >>> <http://localhost:7000/refsByType/0x616022ae8>* >>> >>> 224886 >>> >>> *scala.reflect.io.FileZipArchive$FileEntry$1 >>> <http://localhost:7000/refsByType/0x616022ae8>* >>> >>> 224886 >>> >>> *scala.reflect.io.ZipArchive$DirEntry >>> <http://localhost:7000/refsByType/0x616022ca8>* >>> >>> 10123 >>> >>> *scala.reflect.io.ZipArchive$DirEntry >>> <http://localhost:7000/refsByType/0x616022ca8>* >>> >>> 10123 >>> >>> *sun.nio.cs.UTF_8$Encoder <http://localhost:7000/refsByType/0x61668d160>* >>> >>> 1090 >>> >>> *sun.nio.cs.UTF_8$Encoder <http://localhost:7000/refsByType/0x61668d160>* >>> >>> 1091 >>> >>> *java.util.zip.Inflater <http://localhost:7000/refsByType/0x615fb8d20>* >>> >>> 960 >>> >>> *java.util.zip.Inflater <http://localhost:7000/refsByType/0x615fb8d20>* >>> >>> 960 >>> >>> *org.apache.derby.iapi.services.io.FormatIdInputStream >>> <http://localhost:7000/refsByType/0x61b38f2a0>* >>> >>> 884 >>> >>> *org.apache.derby.iapi.services.io.FormatIdInputStream >>> <http://localhost:7000/refsByType/0x61b38f2a0>* >>> >>> 884 >>> >>> *sun.security.util.DerInputBuffer >>> <http://localhost:7000/refsByType/0x61684fc40>* >>> >>> 745 >>> >>> *sun.security.util.DerInputBuffer >>> <http://localhost:7000/refsByType/0x61684fc40>* >>> >>> 745 >>> >>> *org.apache.derby.iapi.services.io.FormatIdOutputStream >>> <http://localhost:7000/refsByType/0x61b3f5f20>* >>> >>> 479 >>> >>> *org.apache.derby.iapi.services.io.FormatIdOutputStream >>> <http://localhost:7000/refsByType/0x61b3f5f20>* >>> >>> 479 >>> >>> *sun.security.util.ObjectIdentifier >>> <http://localhost:7000/refsByType/0x615d02320>* >>> >>> 466 >>> >>> *sun.security.util.ObjectIdentifier >>> <http://localhost:7000/refsByType/0x615d02320>* >>> >>> 466 >>> >>> *org.apache.derby.iapi.services.io.ArrayInputStream >>> <http://localhost:7000/refsByType/0x61b215910>* >>> >>> 442 >>> >>> *org.apache.derby.iapi.services.io.ArrayInputStream >>> <http://localhost:7000/refsByType/0x61b215910>* >>> >>> 442 >>> >>> *org.apache.derby.iapi.services.io.ArrayOutputStream >>> <http://localhost:7000/refsByType/0x61b3f6000>* >>> >>> 442 >>> >>> *org.apache.derby.iapi.services.io.ArrayOutputStream >>> <http://localhost:7000/refsByType/0x61b3f6000>* >>> >>> 442 >>> >>> *org.apache.derby.iapi.services.io.FormatableBitSet >>> <http://localhost:7000/refsByType/0x617ae17e8>* >>> >>> 364 >>> >>> *org.apache.derby.iapi.services.io.FormatableBitSet >>> <http://localhost:7000/refsByType/0x617ae17e8>* >>> >>> 364 >>> >>> *scala.reflect.internal.pickling.UnPickler$Scan >>> <http://localhost:7000/refsByType/0x6166d6e20>* >>> >>> 282 >>> >>> *scala.reflect.internal.pickling.UnPickler$Scan >>> <http://localhost:7000/refsByType/0x6166d6e20>* >>> >>> 282 >>> >>> *org.apache.derby.impl.store.raw.data.StoredPage >>> <http://localhost:7000/refsByType/0x61ba57d88>* >>> >>> 279 >>> >>> *org.apache.derby.impl.store.raw.data.StoredPage >>> <http://localhost:7000/refsByType/0x61ba57d88>* >>> >>> 279 >>> >>> *java.util.jar.JarFile$JarFileEntry >>> <http://localhost:7000/refsByType/0x615e24618>* >>> >>> 276 >>> >>> *java.util.jar.JarFile$JarFileEntry >>> <http://localhost:7000/refsByType/0x615e24618>* >>> >>> 276 >>> >>> *sun.security.x509.X509CertImpl >>> <http://localhost:7000/refsByType/0x616083cf0>* >>> >>> 166 >>> >>> *sun.security.x509.X509CertImpl >>> <http://localhost:7000/refsByType/0x616083cf0>* >>> >>> 166 >>> >>> *org.apache.derby.impl.store.raw.data.AllocPage >>> <http://localhost:7000/refsByType/0x61b15ef28>* >>> >>> 162 >>> >>> *org.apache.derby.impl.store.raw.data.AllocPage >>> <http://localhost:7000/refsByType/0x61b15ef28>* >>> >>> 162 >>> >>> *org.spark-project.jetty.io.BufferCache$CachedBuffer >>> <http://localhost:7000/refsByType/0x616325310>* >>> >>> 155 >>> >>> *org.spark-project.jetty.io.BufferCache$CachedBuffer >>> <http://localhost:7000/refsByType/0x616325310>* >>> >>> 155 >>> >>> *[[B <http://localhost:7000/refsByType/0x616897178>* >>> >>> 150 >>> >>> *[[B <http://localhost:7000/refsByType/0x616897178>* >>> >>> 150 >>> >>> *org.spark-project.jetty.io.ByteArrayBuffer >>> <http://localhost:7000/refsByType/0x616309b60>* >>> >>> 138 >>> >>> *org.spark-project.jetty.io.ByteArrayBuffer >>> <http://localhost:7000/refsByType/0x616309b60>* >>> >>> 138 >>> >>> *org.apache.thrift.protocol.TBinaryProtocol >>> <http://localhost:7000/refsByType/0x615f91410>* >>> >>> 128 >>> >>> *org.apache.thrift.protocol.TBinaryProtocol >>> <http://localhost:7000/refsByType/0x615f91410>* >>> >>> 128 >>> >>> *scala.reflect.io.VirtualFile >>> <http://localhost:7000/refsByType/0x616022bc8>* >>> >>> 115 >>> >>> *scala.reflect.io.VirtualFile >>> <http://localhost:7000/refsByType/0x616022bc8>* >>> >>> 121 >>> >>> *org.apache.derby.impl.store.raw.data.RAFContainer4 >>> <http://localhost:7000/refsByType/0x61b974dc0>* >>> >>> 100 >>> >>> *org.apache.derby.impl.store.raw.data.RAFContainer4 >>> <http://localhost:7000/refsByType/0x61b974dc0>* >>> >>> 100 >>> >>> *sun.security.x509.KeyIdentifier >>> <http://localhost:7000/refsByType/0x616856000>* >>> >>> 95 >>> >>> *sun.security.x509.KeyIdentifier >>> <http://localhost:7000/refsByType/0x616856000>* >>> >>> 95 >>> >>> *java.lang.Class <http://localhost:7000/refsByType/0x615e60220>* >>> >>> 94 >>> >>> *java.lang.Class <http://localhost:7000/refsByType/0x615e60220>* >>> >>> 94 >>> >>> *sun.security.util.Cache$EqualByteArray >>> <http://localhost:7000/refsByType/0x61688c258>* >>> >>> 84 >>> >>> *sun.security.util.Cache$EqualByteArray >>> <http://localhost:7000/refsByType/0x61688c258>* >>> >>> 84 >>> >>> *sun.security.x509.X509CertInfo >>> <http://localhost:7000/refsByType/0x6163a8398>* >>> >>> 83 >>> >>> *sun.security.util.BitArray >>> <http://localhost:7000/refsByType/0x61683bdb0>* >>> >>> 83 >>> >>> *sun.security.util.BitArray >>> <http://localhost:7000/refsByType/0x61683bdb0>* >>> >>> 83 >>> >>> *sun.security.x509.X509CertInfo >>> <http://localhost:7000/refsByType/0x6163a8398>* >>> >>> 83 >>> >>> *sun.security.rsa.RSAPublicKeyImpl >>> <http://localhost:7000/refsByType/0x6160e75d0>* >>> >>> 80 >>> >>> *sun.security.rsa.RSAPublicKeyImpl >>> <http://localhost:7000/refsByType/0x6160e75d0>* >>> >>> 80 >>> >>> *sun.security.x509.BasicConstraintsExtension >>> <http://localhost:7000/refsByType/0x616568910>* >>> >>> 72 >>> >>> *sun.security.x509.BasicConstraintsExtension >>> <http://localhost:7000/refsByType/0x616568910>* >>> >>> 72 >>> >>> *sun.security.x509.SubjectKeyIdentifierExtension >>> <http://localhost:7000/refsByType/0x616568890>* >>> >>> 65 >>> >>> *sun.security.x509.SubjectKeyIdentifierExtension >>> <http://localhost:7000/refsByType/0x616568890>* >>> >>> 65 >>> >>> *sun.security.x509.KeyUsageExtension >>> <http://localhost:7000/refsByType/0x615e96ec0>* >>> >>> 62 >>> >>> *sun.security.x509.KeyUsageExtension >>> <http://localhost:7000/refsByType/0x615e96ec0>* >>> >>> 62 >>> >>> *sun.security.ec.NamedCurve >>> <http://localhost:7000/refsByType/0x615e28ef0>* >>> >>> 46 >>> >>> *sun.security.ec.NamedCurve >>> <http://localhost:7000/refsByType/0x615e28ef0>* >>> >>> 46 >>> >>> *io.netty.buffer.PoolChunk >>> <http://localhost:7000/refsByType/0x61c0559a0>* >>> >>> 40 >>> >>> *io.netty.buffer.PoolChunk >>> <http://localhost:7000/refsByType/0x61c0559a0>* >>> >>> 40 >>> >>> *scala.tools.nsc.symtab.classfile.Pickler$Pickle >>> <http://localhost:7000/refsByType/0x61637b1a0>* >>> >>> 34 >>> >>> *scala.tools.nsc.symtab.classfile.Pickler$Pickle >>> <http://localhost:7000/refsByType/0x61637b1a0>* >>> >>> 36 >>> >>> *java.nio.HeapByteBuffer <http://localhost:7000/refsByType/0x61672c5b0>* >>> >>> 32 >>> >>> *java.nio.HeapByteBuffer <http://localhost:7000/refsByType/0x61672c5b0>* >>> >>> 32 >>> >>> *sun.security.x509.AuthorityKeyIdentifierExtension >>> <http://localhost:7000/refsByType/0x615f917c8>* >>> >>> 30 >>> >>> *sun.security.x509.AuthorityKeyIdentifierExtension >>> <http://localhost:7000/refsByType/0x615f917c8>* >>> >>> 30 >>> >>> *java.security.cert.PolicyQualifierInfo >>> <http://localhost:7000/refsByType/0x61672f090>* >>> >>> 28 >>> >>> *java.security.cert.PolicyQualifierInfo >>> <http://localhost:7000/refsByType/0x61672f090>* >>> >>> 28 >>> >>> *java.io.ByteArrayOutputStream >>> <http://localhost:7000/refsByType/0x6162c3f98>* >>> >>> 24 >>> >>> *java.io.ByteArrayOutputStream >>> <http://localhost:7000/refsByType/0x6162c3f98>* >>> >>> 24 >>> >>> *java.util.jar.JarVerifier >>> <http://localhost:7000/refsByType/0x615fc7838>* >>> >>> 22 >>> >>> *java.util.jar.JarVerifier >>> <http://localhost:7000/refsByType/0x615fc7838>* >>> >>> 22 >>> >>> *sun.security.x509.CRLDistributionPointsExtension >>> <http://localhost:7000/refsByType/0x616568810>* >>> >>> 17 >>> >>> *sun.security.x509.CRLDistributionPointsExtension >>> <http://localhost:7000/refsByType/0x616568810>* >>> >>> 17 >>> >>> *java.io.BufferedOutputStream >>> <http://localhost:7000/refsByType/0x61623db10>* >>> >>> 13 >>> >>> *java.io.BufferedOutputStream >>> <http://localhost:7000/refsByType/0x61623db10>* >>> >>> 13 >>> >>> *java.net.SocketOutputStream >>> <http://localhost:7000/refsByType/0x61665ded8>* >>> >>> 12 >>> >>> *java.net.SocketOutputStream >>> <http://localhost:7000/refsByType/0x61665ded8>* >>> >>> 12 >>> >>> *sun.security.provider.SHA >>> <http://localhost:7000/refsByType/0x615e756c0>* >>> >>> 12 >>> >>> *sun.security.x509.CertificatePoliciesExtension >>> <http://localhost:7000/refsByType/0x616568710>* >>> >>> 12 >>> >>> *sun.security.x509.CertificatePoliciesExtension >>> <http://localhost:7000/refsByType/0x616568710>* >>> >>> 12 >>> >>> *sun.security.provider.SHA >>> <http://localhost:7000/refsByType/0x615e756c0>* >>> >>> 12 >>> >>> *java.io.BufferedInputStream >>> <http://localhost:7000/refsByType/0x61669a630>* >>> >>> 10 >>> >>> *java.io.BufferedInputStream >>> <http://localhost:7000/refsByType/0x61669a630>* >>> >>> 10 >>> >>> *sun.security.provider.SecureRandom >>> <http://localhost:7000/refsByType/0x615f128c8>* >>> >>> 10 >>> >>> *sun.security.provider.SecureRandom >>> <http://localhost:7000/refsByType/0x615f128c8>* >>> >>> 10 >>> >>> >>> >>> >>> >>> *From:* moon soo Lee [mailto:m...@apache.org] >>> *Sent:* Thursday, February 25, 2016 10:56 PM >>> *To:* users@zeppelin.incubator.apache.org >>> *Subject:* Re: OOM error when run all paragraphs >>> >>> >>> >>> Hi Dafeng, >>> >>> >>> >>> Zeppelin keeps notebooks in memory. >>> >>> That may cause OOM when notebooks has a lot of con >>> >>> >
