Java Heap Space

[Jae Joo]

I am new in Solr and try to use Jitty and example with 13 million records.
During running it, I have the error -
*HTTP ERROR: 500*

Java heap space



java.lang.OutOfMemoryError: Java heap space


Any recommendation? We have a million transactions, so would it be better to
use Tomcat?

Thanks,

Jae

Java heap Space error

[Ameya Aware]

Hi

i am running into java heap space issue. Please see below log.

ERROR - 2014-07-22 11:38:59.370; org.apache.solr.common.SolrException;
null:java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space
at
org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:790)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:439)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:368)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:636)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.OutOfMemoryError: Java heap space
at org.apache.solr.common.util.JavaBinCodec.writeStr(JavaBinCodec.java:567)
at
org.apache.solr.common.util.JavaBinCodec.writePrimitive(JavaBinCodec.java:646)
at
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:240)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:153)
at
org.apache.solr.common.util.JavaBinCodec.writeSolrInputDocument(JavaBinCodec.java:409)
at org.apache.solr.update.TransactionLog.write(TransactionLog.java:353)
at org.apache.solr.update.UpdateLog.add(UpdateLog.java:397)
at org.apache.solr.update.UpdateLog.add(UpdateLog.java:382)
at
org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:255)
at
org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:160)
at
org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69)
at
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:704)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:858)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:557)
at
org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100)
at
org.apache.solr.handler.extraction.ExtractingDocumentLoader.doAdd(ExtractingDocumentLoader.java:121)
at
org.apache.solr.handler.extraction.ExtractingDocumentLoader.addDoc(ExtractingDocumentLoader.java:126)
at
org.apache.solr.handler.extraction.ExtractingDocumentLoader.load(ExtractingDocumentLoader.java:228)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at
org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:241)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1952)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:774)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
at

Java heap space error

[Ameya Aware]

Hi

I am in process of indexing around 2,00,000 documents.

I have increase java jeap space to 4 GB using below command :

java -Xmx4096M -Xms4096M -jar start.jar

Still after indexing around 15000 documents it gives java heap space error
again.


Any fix for this?

Thanks,
Ameya

DataImporter : Java heap space

[Mani Kumar]

Hi All,
I am trying to setup a Solr instance on my macbook.

I get following errors when m trying to do a full db import ... please help
me on this

Apr 13, 2009 11:53:28 PM org.apache.solr.handler.dataimport.JdbcDataSource$1
call
INFO: Creating a connection for entity slideshow with URL:
jdbc:mysql://localhost/mydb_development
Apr 13, 2009 11:53:29 PM org.apache.solr.handler.dataimport.JdbcDataSource$1
call
INFO: Time taken for getConnection(): 319
Apr 13, 2009 11:53:32 PM org.apache.solr.handler.dataimport.DataImporter
doFullImport
SEVERE: Full Import failed
org.apache.solr.handler.dataimport.DataImportHandlerException:
java.lang.OutOfMemoryError: Java heap space
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:400)
at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:221)
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:164)
at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:312)
at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:370)
at
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:351)
Caused by: java.lang.OutOfMemoryError: Java heap space
at com.mysql.jdbc.Buffer.(Buffer.java:58)
at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1444)
at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:2840)


My Java version
$ java -version
java version "1.5.0_16"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_16-b06-284)
Java HotSpot(TM) Client VM (build 1.5.0_16-133, mixed mode, sharing)


Is that i need to install a new java version?
my db is also very huge ~15 GB

please do the need full ...

thanks
mani kumar

Re: Java Heap Space

[Thierry Collogne]

Hello,

You need to start your jetty or tomcat server with higher vm memory
settings.

On this page you can find some explanation

http://www.caucho.com/resin-3.0/performance/jvm-tuning.xtp

The 2 parameters that are important are -Xms and -Xmx.

So instead of starting jetty with

java -jar start.jar

you should use something like

java -Xmx512M -Xms512M -jar start.jar

This will start the server with an initital memory of 512 MB and a maximum
of 512 MB. If that isn't enough, you can use higher values.

Hope this helps,

Thierry Collogne

On 27/07/07, Jae Joo <[EMAIL PROTECTED]> wrote:
>
> I am new in Solr and try to use Jitty and example with 13 million records.
> During running it, I have the error -
> *HTTP ERROR: 500*
>
> Java heap space
>
>
>
> java.lang.OutOfMemoryError: Java heap space
>
>
> Any recommendation? We have a million transactions, so would it be better
> to
> use Tomcat?
>
> Thanks,
>
> Jae
>

Re: Java Heap Space

[Chris Hostetter]

: I am new in Solr and try to use Jitty and example with 13 million records.
: During running it, I have the error -

: java.lang.OutOfMemoryError: Java heap space

: Any recommendation? We have a million transactions, so would it be better to
: use Tomcat?

millions of records takes up memory.  what is your heap size curently set
to?  did you try increasing it?

you should also consider searching the mailing list archives for "heap
tuning" or "memory" or even "OutOfMemoryError"



-Hoss

Re: Java heap space

[Chris Hostetter]

: I'm currently testing a large index with more than 10 million
: documents and 24 fields, using the example installation with
: Jetty.
: When deleting or updateing documents from the index or doing
: search queries I get "Java heap space" error messages like
: this (in this case while trying to delete documents):
...
: In both cases obviously the server ran out of heap space.
: I'm wondering what I can do to prevent this. I started the
: server using the java options "-server -Xmx1024m".
:
: Does anybody else have problems with heap space using a large
: index? Is there anything I can do against this?

How big is your physical index directory on disk?

Generally speaking, there isn't much you can do to improve upon the memory
needs of a Lucene index that Solr isn't allready doing: Reuse a single
IndexSearcher as much as possible.

Your best bet is to allocate as much ram to the server as you can.
Depending on how full your caches are, and what hitratios you are getting
(the "STATISTICS" link from the Admin screen will tell you) you might want
to make some of them smaller to reduce the amount of RAM Solr uses for
them.

>From an acctual index standpoint, if you don't care about doc/field boosts
of lengthNorms, then the omitNorm="true" option on your fields (or
fieldtypes) will help save one byte per document per field you use it on.


-Hoss

Re: Java heap space

[Marcus Stratmann]

Chris Hostetter wrote:

How big is your physical index directory on disk?

It's about 2.9G now.
Is there a direct connection between size of index and usage of ram?


Your best bet is to allocate as much ram to the server as you can.
Depending on how full your caches are, and what hitratios you are getting
(the "STATISTICS" link from the Admin screen will tell you) you might want
to make some of them smaller to reduce the amount of RAM Solr uses for
them.

Hm, after disabling all caches I still get OutOfMemoryErrors.
All I do currently while testing is to delete documents. No searching or 
inserting. Typically after deleting about 20,000 documents the server 
throws the first error message.



From an acctual index standpoint, if you don't care about doc/field boosts

of lengthNorms, then the omitNorm="true" option on your fields (or
fieldtypes) will help save one byte per document per field you use it on.
That is something I could test, though I think this won't significantly 
change the size of the index.


One thing that appears suspicious to me is that everything went fine as 
long as the number of documents was below 10 million. Problems started 
when this limit was exceeded. But maybe this is just a coincidence.


Marcus

Re: Java heap space

[Chris Hostetter]

: > How big is your physical index directory on disk?
: It's about 2.9G now.
: Is there a direct connection between size of index and usage of ram?

Generally yes.  Lucene loads a lot of your index into memory ... not
neccessarily the "stored" fields, but quite a bit of the index structure
needed forsearching ... not to mention things like the FieldCache if you
start sorting by specific fields.

You may want to consult the lucene user list to get more advice about the
minimum amount of RAM recommended just for using a lucene index of size
___G in a JVM ... personally, I've always made sure my JVM was
allocated a max Heap size at least twice as big as my acctual index ...
mainly a paranoia thing on my part.

: Hm, after disabling all caches I still get OutOfMemoryErrors.
: All I do currently while testing is to delete documents. No searching or
: inserting. Typically after deleting about 20,000 documents the server
: throws the first error message.

interesting .. are you getting the OutOfMemory on an actual delete
operation or when doing a commit after executing some deletes?

part of the problem may be that under the covers, any delete involves
doing a query (even if oyou are deleting by uniqueKey, that's implimented
as a delete by Term, which requires iterating over a TermEnum to find the
relevent document, and if your index is big enough, loading that TermEnum
and may be the cause of your OOM.

: One thing that appears suspicious to me is that everything went fine as
: long as the number of documents was below 10 million. Problems started
: when this limit was exceeded. But maybe this is just a coincidence.

Maybe, maybe not ... what options are you using in your solrconfig.xml's
indexDefaults and mainIndex blocks? ... 10 million documents could be the
magic point at which your mergeFactor triggers the merging of several
large segments into one uber segment -- which may be big enough to cause
an OOM when the IndexReader tries to open it.

(Doug, Yonik, and Erik understand the underlying lucene memory usange
better then i do, hopefully they'll chime in with some advice)


-Hoss

Re: Java heap space

[Marcus Stratmann]

Chris Hostetter wrote:
> interesting .. are you getting the OutOfMemory on an actual delete
> operation or when doing a commit after executing some deletes?

Yes, on a delete operation. I'm not doing any commits until the end of
all delete operations.
After reading this I was curious if using commits during deleting would
have any effect. So I tested doing a commit after 10,000 deletes at a
time (which, I know, is not recommended). But that simply didn't change
anything.

Meanwhile I found out that I can gain 10,000 documents more to delete
(before getting an OOM) by increasing the heap space by 500M.
Unfortunately we need to delete about 200,000 documents on each update
which would need 10G to be added to the heap space. Not to speak of
the same number of inserts.


> part of the problem may be that under the covers, any delete involves
> doing a query (even if oyou are deleting by uniqueKey, that's implimented
> s a delete by Term, which requires iterating over a TermEnum to find the
> relevent document, and if your index is big enough, loading that TermEnum
> and may be the cause of your OOM.

Yes, I thought so, too. And in fact I get OOM even if I just submit search
queries.


> Maybe, maybe not ... what options are you using in your solrconfig.xml's
> indexDefaults and mainIndex blocks?

I adopted the default values from the example installation which looked
quite reasonable to me.


> ... 10 million documents could be the
> magic point at which your mergeFactor triggers the merging of several
> large segments into one uber segment -- which may be big enough to cause
> an OOM when the IndexReader tries to open it.

Yes, I'm using the default mergeFactor of 10 and as 10 million is 10^7
this is what appeared suspicious to me.
Is it right, that the mergeFactor connot be changed once the index has
been built?

Marcus

Re: Java heap space

[Yonik Seeley]

On 4/29/06, Marcus Stratmann <[EMAIL PROTECTED]> wrote:

Yes, on a delete operation. I'm not doing any commits until the end of
all delete operations.


I assume this is a delete-by-id and not a delete-by-query?  They work
very differently.

There is some state stored for each pending delete-by-id... there is
a HashMap with an entry for each id that needs to be
deleted.  This state shouldn't be that large though.

If fact, delete-by-id does nothing with a Lucene index at all until 


After reading this I was curious if using commits during deleting would
have any effect. So I tested doing a commit after 10,000 deletes at a
time (which, I know, is not recommended). But that simply didn't change
anything.


Strange... that suggests it's not the state kept in the HaspMap.


Meanwhile I found out that I can gain 10,000 documents more to delete
(before getting an OOM) by increasing the heap space by 500M.


Something else is going on.


Unfortunately we need to delete about 200,000 documents on each update
which would need 10G to be added to the heap space. Not to speak of
the same number of inserts.


If you are first deleting so you can re-add a newer version of the
document, you don't need too... overwriting older documents based on
the uniqueKeyField is something Solr does for you!


Yes, I thought so, too. And in fact I get OOM even if I just submit search
queries.


Is it possible to use a profiler to see where all the memory is going?
It sounds like you may have uncovered a memory leak somewhere.
Also what OS, what JVM, what appserver are you using?


-Yonik

Re: Java heap space

[Marcus Stratmann]

Yonik Seeley wrote:

Yes, on a delete operation. I'm not doing any commits until the end of
all delete operations.

I assume this is a delete-by-id and not a delete-by-query?  They work
very differently.


Yes, all queries are delete-by-id.



If you are first deleting so you can re-add a newer version of the
document, you don't need too... overwriting older documents based on
the uniqueKeyField is something Solr does for you!


Yes, I know. But the articles in our (sql-)database get new IDs when 
they are changed so they need to be deleted an re-inserted into the index.




Is it possible to use a profiler to see where all the memory is going?
It sounds like you may have uncovered a memory leak somewhere.


I'm not that experienced concerning Java, but maybe if you give me some 
advice I'm glad if I can help. So far I had a quick look at JMP once but 
that's all.

Don't hesitate to write me a PM on that subject.



Also what OS, what JVM, what appserver are you using?

OS: Linux (Debian GNU/Linux i686)
JVM: Java HotSpot(TM) Server VM (build 1.5.0_06-b05, mixed mode) of 
Sun's JDK 5.0.
Currently I'm using the Jetty installation from the solr nightly builds 
for test purposes.


Marcus

Re: Java heap space

[Chris Hostetter]

: > If you are first deleting so you can re-add a newer version of the
: > document, you don't need too... overwriting older documents based on
: > the uniqueKeyField is something Solr does for you!
:
: Yes, I know. But the articles in our (sql-)database get new IDs when
: they are changed so they need to be deleted an re-inserted into the index.

this is off the subject of the heap space issue ... but if the id changes,
then maybe it shouldn't be the uniqueId of your index? .. your code must
have someone of recognizing that article B with id 222 is a changed
version of article A with id 111 (otherwise how would you know to delete
111 when you insert 222?) ..whatever that mechanism is, perhaps it should
determine your uniqueKey?


-Hoss

Re: Java heap space

[Marcus Stratmann]

Chris Hostetter wrote:
> this is off the subject of the heap space issue ... but if the id changes,
> then maybe it shouldn't be the uniqueId of your index? .. your code must
> have someone of recognizing that article B with id 222 is a changed
> version of article A with id 111 (otherwise how would you know to delete
> 111 when you insert 222?) ..whatever that mechanism is, perhaps it should
> determine your uniqueKey?

No, there is no "key" or something that reveals a relation between new
article B and old article A. After B is inserted and A is deleted, all
of A's existence is gone and we do not even know that B is A's
"successor". Changes are simply kept in a table which tells the system
which IDs to delete and which new (or changed) articles to insert,
automatically giving them new IDs. I know this may not be (or at least
sound) perfect and it is not the way things are handled normally. But
this works fine for our needs. We gather information about changes to
our data during the day and apply them on a nightly update (which, I
know, does not imply that IDs have to change).
So, yes, I'm sure I got the right uniqueKey. ;-)

Marcus

Re: Java heap space

[Marcus Stratmann]

Hello,

deleting or updating documents is still not possible for me so now I 
tried to built a completely new index. Unfortunately this didn't work 
either. Now I'm getting OOM after inserting slightly more than 20,000 
documents to the new index.


To me this looks as if a bug has been introduced since the revision of 
about 13th of april. To check this out, I looked for old builds but 
there seem to be nightly-builds of the last four days only. Okay, so 
next thing I tried was to get the code via svn. Unfortunately the code 
does not compile ("package junit.framework does not exist"). I found out 
that the last version I was able to complie was revision 393080 
(2006-04-10). So I was neither able to get back the last (for me) 
working revision nor to find out which revision this actually was.
Sorry, I would really like to help, but at the moment it seems Murphy is 
striking.


Thanks,
Marcus

Re: Java heap space

[Yonik Seeley]

Hi Marcus,

Is your problem reproducable with a test case you can share?
You could also try a different app-server like Tomcat to see if that
makes a difference.

What type is your id field defined to be?

-Yonik

On 5/3/06, Marcus Stratmann <[EMAIL PROTECTED]> wrote:

Hello,

deleting or updating documents is still not possible for me so now I
tried to built a completely new index. Unfortunately this didn't work
either. Now I'm getting OOM after inserting slightly more than 20,000
documents to the new index.

To me this looks as if a bug has been introduced since the revision of
about 13th of april. To check this out, I looked for old builds but
there seem to be nightly-builds of the last four days only. Okay, so
next thing I tried was to get the code via svn. Unfortunately the code
does not compile ("package junit.framework does not exist"). I found out
that the last version I was able to complie was revision 393080
(2006-04-10). So I was neither able to get back the last (for me)
working revision nor to find out which revision this actually was.
Sorry, I would really like to help, but at the moment it seems Murphy is
striking.

Thanks,
Marcus

Re: Java heap space

[Marcus Stratmann]

Yonik Seeley wrote:

Is your problem reproducable with a test case you can share?

Well, you can get the configuration files. If you ask for the data, this
could be a problem, since this is "real" data from our production
database. The amount of data needed could be another problem.


You could also try a different app-server like Tomcat to see if that
makes a difference.

This is another part of the problem because currently Tomcat won't work
with solr in my environment (a debian linux installation).


What type is your id field defined to be?


with slong defined by

as in the sample schema.xml.

Marcus

Re: Java heap space

[Chris Hostetter]

: next thing I tried was to get the code via svn. Unfortunately the code
: does not compile ("package junit.framework does not exist"). I found out

This is because building a full Solr distribution from scratch requires
that you have JUnit.  Bt it is not required to run Solr.

: > Is your problem reproducable with a test case you can share?
: Well, you can get the configuration files. If you ask for the data, this
: could be a problem, since this is "real" data from our production
: database. The amount of data needed could be another problem.

What about generating random data dynamicly?

If you can submit a Jira issue, containing...

  1) Solr the config files you use.
  2) information on which ServletContainer you are testing this on
 (Tomcat, Jetty, etc..)
  3) information on how that servlet container is configured and run
 (ie: the container's config files ... or if you are just using the
 Jetty start.jar/configs that come in the Solr example say that)
  3) an app written in any common language (java, bash, perl, python,
 whatever) that generates and POSTs enough "random" documents to Solr
 to trigger your bug

...then other people can try to reproduce your error.

(Hopefully the bug you are encountering is specific to the configs, or the
Solr code, or the container -- and not a result of something strange in
your specific data, otherwise randomly generating fake data won't help).


-Hoss

Re: Java heap space

[Yonik Seeley]

I just tried sending in 100,000 deletes and it didn't cause a problem:
the memory grew from 22M to 30M.

Random thought: perhaps it has something to do with how you are
sending your requests?
If the client creates a new connection for each request, but doesn't
send the Connection:close header or close the connection after use,
these persistent connections can cause new threads to be created in
the app-server.

You can do a thread-dump with the latest nightly build to see if you
are accumulating too many threads in the appserver while the deletes
are going on (prob shouldn't be more than 20):
http://localhost:8983/solr/admin/threaddump.jsp

Here's the python script I used to try the 100,000 deletes... maybe
you can try it in your environment to see if it reproduces your memory
problem or not.  If it doesn't, look to your client code for possible
bugs.

-Yonik

- python script --
import httplib

class SolrConnection:
 def __init__(self, host='localhost:8983', solrBase='/solr'):
   self.host = host
   self.solrBase = solrBase
   #a socket is not really opened at this point.
   self.conn = httplib.HTTPConnection(self.host)
   self.conn.set_debuglevel(100)

 def doUpdateXML(self, request):
   self.conn.request('POST', self.solrBase+'/update', request)
   rsp = self.conn.getresponse()
   print rsp.status, rsp.reason
   data = rsp.read()
   print "data=",data

 def delete(self, id):
   xstr = ''+id+''
   self.doUpdateXML(xstr)

c = SolrConnection()
for i in range(10):
 c.delete(str(i))

Re: Java heap space

[Marcus Stratmann]

Chris Hostetter wrote:

This is because building a full Solr distribution from scratch requires
that you have JUnit.  Bt it is not required to run Solr.

Ah, I see. That was a very valuable hint for me.
I was able now to compile an older revision (393957). Testing this 
revision I was able to delete more than 600,000 documents without problems.
From my point of view it looks like this: Revision 393957 works while 
the latest revision cause problems. I don't know what part of the 
distribution causes the problems but I will try to find out. I think a 
good start would be to find out which was the first revision not working 
for me. Maybe this would be enough information for you to find out what 
had been changed at this point and what causes the problems.
I will also try just to change the solr.war to check if maybe Jetty is 
responsible for the OOM.


I'll post a report when I have some results.

Marcus

Re: Java heap space

[Yonik Seeley]

On 5/3/06, Yonik Seeley <[EMAIL PROTECTED]> wrote:

I just tried sending in 100,000 deletes and it didn't cause a problem:
the memory grew from 22M to 30M.

Random thought: perhaps it has something to do with how you are
sending your requests?


Yep, I was able to reproduce a memory problem w/ Jetty on Linux when
using non-persistent connections (closed after each request).  The
same 100,000 deletes blew up the JVM to 1GB heap.

So this looks like it could be a Jetty problem (shame on me for using a beta).
I'm still not quite sure what changed in Solr that could make it
appear in later version and not in earlier versions though... the
version of Jetty is the same.

-Yonik

Re: Java heap space

[Yonik Seeley]

I verified that Tomcat 5.5.17 doesn't experience this problem.

-Yonik

On 5/4/06, Yonik Seeley <[EMAIL PROTECTED]> wrote:

On 5/3/06, Yonik Seeley <[EMAIL PROTECTED]> wrote:
> I just tried sending in 100,000 deletes and it didn't cause a problem:
> the memory grew from 22M to 30M.
>
> Random thought: perhaps it has something to do with how you are
> sending your requests?

Yep, I was able to reproduce a memory problem w/ Jetty on Linux when
using non-persistent connections (closed after each request).  The
same 100,000 deletes blew up the JVM to 1GB heap.

So this looks like it could be a Jetty problem (shame on me for using a beta).
I'm still not quite sure what changed in Solr that could make it
appear in later version and not in earlier versions though... the
version of Jetty is the same.

Re: Java heap space

[Bill Au]

There seems to be a fair number of folks using the jetty with the example
app
as oppose to using Solr with their own appserver.  So I think it is best to
use a stable version of Jetty instead of the beta.  If no one objects, I can
go ahead and take care of this.

Bill

On 5/4/06, Yonik Seeley <[EMAIL PROTECTED]> wrote:


I verified that Tomcat 5.5.17 doesn't experience this problem.

-Yonik

On 5/4/06, Yonik Seeley <[EMAIL PROTECTED]> wrote:
> On 5/3/06, Yonik Seeley <[EMAIL PROTECTED]> wrote:
> > I just tried sending in 100,000 deletes and it didn't cause a problem:
> > the memory grew from 22M to 30M.
> >
> > Random thought: perhaps it has something to do with how you are
> > sending your requests?
>
> Yep, I was able to reproduce a memory problem w/ Jetty on Linux when
> using non-persistent connections (closed after each request).  The
> same 100,000 deletes blew up the JVM to 1GB heap.
>
> So this looks like it could be a Jetty problem (shame on me for using a
beta).
> I'm still not quite sure what changed in Solr that could make it
> appear in later version and not in earlier versions though... the
> version of Jetty is the same.

Re: Java heap space

[Erik Hatcher]

Along these lines, locally I've been using the latest stable version  
of Jetty and it has worked fine, but I did see an "out of memory"  
exception the other day but have not seen it since so I'm not sure  
what caused it.


Moving to Tomcat, as long as we can configure it to be as lightweight  
as possible, is quite fine to me as well.


Erik


On May 5, 2006, at 12:12 PM, Bill Au wrote:

There seems to be a fair number of folks using the jetty with the  
example

app
as oppose to using Solr with their own appserver.  So I think it is  
best to
use a stable version of Jetty instead of the beta.  If no one  
objects, I can

go ahead and take care of this.

Bill

On 5/4/06, Yonik Seeley <[EMAIL PROTECTED]> wrote:


I verified that Tomcat 5.5.17 doesn't experience this problem.

-Yonik

On 5/4/06, Yonik Seeley <[EMAIL PROTECTED]> wrote:
> On 5/3/06, Yonik Seeley <[EMAIL PROTECTED]> wrote:
> > I just tried sending in 100,000 deletes and it didn't cause a  
problem:

> > the memory grew from 22M to 30M.
> >
> > Random thought: perhaps it has something to do with how you are
> > sending your requests?
>
> Yep, I was able to reproduce a memory problem w/ Jetty on Linux  
when

> using non-persistent connections (closed after each request).  The
> same 100,000 deletes blew up the JVM to 1GB heap.
>
> So this looks like it could be a Jetty problem (shame on me for  
using a

beta).
> I'm still not quite sure what changed in Solr that could make it
> appear in later version and not in earlier versions though... the
> version of Jetty is the same.

Re: Java heap space

[Bill Au]

I was able to produce an OutOfMemoryError using Yonik's python script with
Jetty 6.
I was not able to do so with Jetty 5.1.11RC0, the latest stable version.  So
that's the
version of Jetty with which I will downgrade the Solr example app to.

Bill

On 5/5/06, Erik Hatcher <[EMAIL PROTECTED]> wrote:


Along these lines, locally I've been using the latest stable version
of Jetty and it has worked fine, but I did see an "out of memory"
exception the other day but have not seen it since so I'm not sure
what caused it.

Moving to Tomcat, as long as we can configure it to be as lightweight
as possible, is quite fine to me as well.

Erik


On May 5, 2006, at 12:12 PM, Bill Au wrote:

> There seems to be a fair number of folks using the jetty with the
> example
> app
> as oppose to using Solr with their own appserver.  So I think it is
> best to
> use a stable version of Jetty instead of the beta.  If no one
> objects, I can
> go ahead and take care of this.
>
> Bill
>
> On 5/4/06, Yonik Seeley <[EMAIL PROTECTED]> wrote:
>>
>> I verified that Tomcat 5.5.17 doesn't experience this problem.
>>
>> -Yonik
>>
>> On 5/4/06, Yonik Seeley <[EMAIL PROTECTED]> wrote:
>> > On 5/3/06, Yonik Seeley <[EMAIL PROTECTED]> wrote:
>> > > I just tried sending in 100,000 deletes and it didn't cause a
>> problem:
>> > > the memory grew from 22M to 30M.
>> > >
>> > > Random thought: perhaps it has something to do with how you are
>> > > sending your requests?
>> >
>> > Yep, I was able to reproduce a memory problem w/ Jetty on Linux
>> when
>> > using non-persistent connections (closed after each request).  The
>> > same 100,000 deletes blew up the JVM to 1GB heap.
>> >
>> > So this looks like it could be a Jetty problem (shame on me for
>> using a
>> beta).
>> > I'm still not quite sure what changed in Solr that could make it
>> > appear in later version and not in earlier versions though... the
>> > version of Jetty is the same.
>>

Re: Java heap space

[Bill Au]

FYI, I have just committed the a

On 5/8/06, Bill Au <[EMAIL PROTECTED]> wrote:


I was able to produce an OutOfMemoryError using Yonik's python script with
Jetty 6.
I was not able to do so with Jetty 5.1.11RC0, the latest stable version.
So that's the
version of Jetty with which I will downgrade the Solr example app to.

Bill


On 5/5/06, Erik Hatcher <[EMAIL PROTECTED]> wrote:
>
> Along these lines, locally I've been using the latest stable version
> of Jetty and it has worked fine, but I did see an "out of memory"
> exception the other day but have not seen it since so I'm not sure
> what caused it.
>
> Moving to Tomcat, as long as we can configure it to be as lightweight
> as possible, is quite fine to me as well.
>
> Erik
>
>
> On May 5, 2006, at 12:12 PM, Bill Au wrote:
>
> > There seems to be a fair number of folks using the jetty with the
> > example
> > app
> > as oppose to using Solr with their own appserver.  So I think it is
> > best to
> > use a stable version of Jetty instead of the beta.  If no one
> > objects, I can
> > go ahead and take care of this.
> >
> > Bill
> >
> > On 5/4/06, Yonik Seeley <[EMAIL PROTECTED]> wrote:
> >>
> >> I verified that Tomcat 5.5.17 doesn't experience this problem.
> >>
> >> -Yonik
> >>
> >> On 5/4/06, Yonik Seeley <[EMAIL PROTECTED]> wrote:
> >> > On 5/3/06, Yonik Seeley < [EMAIL PROTECTED]> wrote:
> >> > > I just tried sending in 100,000 deletes and it didn't cause a
> >> problem:
> >> > > the memory grew from 22M to 30M.
> >> > >
> >> > > Random thought: perhaps it has something to do with how you are
> >> > > sending your requests?
> >> >
> >> > Yep, I was able to reproduce a memory problem w/ Jetty on Linux
> >> when
> >> > using non-persistent connections (closed after each request).  The
> >> > same 100,000 deletes blew up the JVM to 1GB heap.
> >> >
> >> > So this looks like it could be a Jetty problem (shame on me for
> >> using a
> >> beta).
> >> > I'm still not quite sure what changed in Solr that could make it
> >> > appear in later version and not in earlier versions though... the
> >> > version of Jetty is the same.
> >>
>
>

Re: Java heap space

[Bill Au]

Sorry, hit the wrong key before...

FYI, I have just committed all the changes related to the Jetty downgrade
into SVN.
Let me know if you notice anything problems.

Bill

On 5/9/06, Bill Au <[EMAIL PROTECTED]> wrote:


FYI, I have just committed the a


On 5/8/06, Bill Au <[EMAIL PROTECTED]> wrote:
>
> I was able to produce an OutOfMemoryError using Yonik's python script
> with Jetty 6.
> I was not able to do so with Jetty 5.1.11RC0, the latest stable
> version.  So that's the
> version of Jetty with which I will downgrade the Solr example app to.
>
> Bill
>
>
> On 5/5/06, Erik Hatcher < [EMAIL PROTECTED]> wrote:
> >
> > Along these lines, locally I've been using the latest stable version
> > of Jetty and it has worked fine, but I did see an "out of memory"
> > exception the other day but have not seen it since so I'm not sure
> > what caused it.
> >
> > Moving to Tomcat, as long as we can configure it to be as lightweight
> > as possible, is quite fine to me as well.
> >
> > Erik
> >
> >
> > On May 5, 2006, at 12:12 PM, Bill Au wrote:
> >
> > > There seems to be a fair number of folks using the jetty with the
> > > example
> > > app
> > > as oppose to using Solr with their own appserver.  So I think it is
> > > best to
> > > use a stable version of Jetty instead of the beta.  If no one
> > > objects, I can
> > > go ahead and take care of this.
> > >
> > > Bill
> > >
> > > On 5/4/06, Yonik Seeley <[EMAIL PROTECTED]> wrote:
> > >>
> > >> I verified that Tomcat 5.5.17 doesn't experience this problem.
> > >>
> > >> -Yonik
> > >>
> > >> On 5/4/06, Yonik Seeley <[EMAIL PROTECTED]> wrote:
> > >> > On 5/3/06, Yonik Seeley < [EMAIL PROTECTED]> wrote:
> > >> > > I just tried sending in 100,000 deletes and it didn't cause a
> > >> problem:
> > >> > > the memory grew from 22M to 30M.
> > >> > >
> > >> > > Random thought: perhaps it has something to do with how you are
> >
> > >> > > sending your requests?
> > >> >
> > >> > Yep, I was able to reproduce a memory problem w/ Jetty on Linux
> > >> when
> > >> > using non-persistent connections (closed after each
> > request).  The
> > >> > same 100,000 deletes blew up the JVM to 1GB heap.
> > >> >
> > >> > So this looks like it could be a Jetty problem (shame on me for
> > >> using a
> > >> beta).
> > >> > I'm still not quite sure what changed in Solr that could make it
> > >> > appear in later version and not in earlier versions though... the
> > >> > version of Jetty is the same.
> > >>
> >
> >
>

Re: Java heap space

[Marcus Stratmann]

On 5/4/06, I wrote:
> From my point of view it looks like this: Revision 393957 works while
> the latest revision cause problems. I don't know what part of the 
> distribution causes the problems but I will try to find out. I think a
> good start would be to find out which was the first revision not working
> for me. Maybe this would be enough information for you to find out what
> had been changed at this point and what causes the problems.
(As a reminder, this was a problem with Jetty.)
Unfortunately I was not able to figure out what was going on. I
compiled some newer revisions from may but my problem with deleting a
huge amount of documents did not appear again. Maybe this is because I
changed the configuration a bit, adding "omitNorms=true" for some
fields.

Meanwhile I switched over to tomcat 5.5 as application server and
things seem to go fine now. The only situation I get OutOfMemory
errors is after an optimize when the server performs an auto-warming
of the cahces:
SEVERE: Error during auto-warming of key:[EMAIL 
PROTECTED]:java.lang.OutOfMemoryError: Java heap space
(from the tomcat log)
But nevertheless the server seems to run stable now with nearly 11
million documents.

Thanks to all the friendly people helping me so far!
Marcus

Re: Java heap space

[Yonik Seeley]

On 5/15/06, Marcus Stratmann <[EMAIL PROTECTED]> wrote:


The only situation I get OutOfMemory
errors is after an optimize when the server performs an auto-warming
of the cahces:


A single filter that is big enough to be represented as a bitset
(>3000 in general) will take up 1.3MB

Some ways to help memory:
 - increase the heap size ;-)
 - make sure you don't have autowarming for more than one searcher
happening at a time.  If this happens, you should see something in
your logs like "PERFORMANCE WARNING: Overlapping onDeckSearchers=2"
 - do omitNorms on every field you can... every field with norms will
take up 1 byte per document (11MB in your case)
 - make caches smaller if you can survive the performance hit... A
single filter that is represented as a BitSet will take up 1.3MB for
11M docs (and bigger in the case that maxDocs is larger tha numDocs
because of deletions).


-Yonik

java.lang.OutOfMemoryError: Java heap space

[Mandava, Rahul]

I am using SOLR version 6.6.0 and the heap size is set to 512 MB, I believe 
which is default. We do have almost 10 million documents in the index, we do 
perform frequent updates (we are doing soft commit on every update: heap issue 
was seen with and without soft commit) to the index and obviously search 
heavily. We have experienced Heap space out of memory exception twice so far in 
the whole year span since we started using SOLR. Since we are just using 
default value for heap size, I am thinking to increase it and I do know that 
high heap size can slow down the performance due to GC pauses. As we can't 
really come up with an ideal number that can work with any scenario, I want to 
increase it to just 1gb only.

I did some reading around this, in which I learned that there can be lot of 
parameters that contribute to this issue and there is no perfect way to address 
this. And also read that increasing heap size above 2gb is where we definitely 
be in the danger zone, since I am thinking to increase it to just 1gb and if I 
monitor the consumption on a daily basis for a while, I should be good and 
resolve the heap memory issue. Is that a safe assumption ??

Does anyone has experienced similar issue ?? Any thoughts or suggestions ??


Below are heap usages if it helps. Usage was almost 490 mb, which makes me feel 
with the load we have 512 mb is not enough and should be good if I increase it 
to 1gb.


[cid:image002.png@01D54174.BF7511F0]

[cid:image001.png@01D54174.6D329160]



Thanks

Solr java.lang.OutOfMemoryError: Java heap space

[Ajinkya Kale]

Hi,

I am trying to retrieve all the documents from a solr index in a batched
manner.
I have 100M documents. I am retrieving them using the method proposed here
https://nowontap.wordpress.com/2014/04/04/solr-exporting-an-index-to-an-external-file/
I am dumping 10M document splits in each file. I get "OutOfMemoryError" if
start is at 50M. I get the same error even if rows=10 for start=50M.
Curl on start=0 rows=50M in one go works fine too. But things go bad when
start is at 50M.
My Solr version is 4.4.0.

Caused by: java.lang.OutOfMemoryError: Java heap space
at
org.apache.lucene.search.TopDocsCollector.topDocs(TopDocsCollector.java:146)
at
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1502)
at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1363)
at
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:474)
at
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:434)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1904)

--aj

Re: Java heap Space error

[Shawn Heisey]

On 7/22/2014 11:37 AM, Ameya Aware wrote:
> i am running into java heap space issue. Please see below log.

All we have here is an out of memory exception.  It is impossible to
know *why* you are out of memory from the exception.  With enough
investigation, we could determine the area of code where the error
occurred, but that doesn't say anything at all about what allocated all
the memory, it's simply the final allocation that pushed it over the edge.

Here is some generic information about Solr and high heap usage:

http://wiki.apache.org/solr/SolrPerformanceProblems#Java_Heap

Thanks,
Shawn

Re: Java heap Space error

[Ameya Aware]

So can i come over this exception by increasing heap size somewhere?

Thanks,
Ameya


On Tue, Jul 22, 2014 at 2:00 PM, Shawn Heisey  wrote:

> On 7/22/2014 11:37 AM, Ameya Aware wrote:
> > i am running into java heap space issue. Please see below log.
>
> All we have here is an out of memory exception.  It is impossible to
> know *why* you are out of memory from the exception.  With enough
> investigation, we could determine the area of code where the error
> occurred, but that doesn't say anything at all about what allocated all
> the memory, it's simply the final allocation that pushed it over the edge.
>
> Here is some generic information about Solr and high heap usage:
>
> http://wiki.apache.org/solr/SolrPerformanceProblems#Java_Heap
>
> Thanks,
> Shawn
>
>

Re: Java heap Space error

[Rafał Kuć]

Hello!

Yes, just edit your Jetty configuration file and add -Xmx and -Xms
parameters. For example, the file you may be looking at it
/etc/default/jetty.

-- 
Regards,
 Rafał Kuć
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/


> So can i come over this exception by increasing heap size somewhere?

> Thanks,
> Ameya


> On Tue, Jul 22, 2014 at 2:00 PM, Shawn Heisey  wrote:

>> On 7/22/2014 11:37 AM, Ameya Aware wrote:
>> > i am running into java heap space issue. Please see below log.
>>
>> All we have here is an out of memory exception.  It is impossible to
>> know *why* you are out of memory from the exception.  With enough
>> investigation, we could determine the area of code where the error
>> occurred, but that doesn't say anything at all about what allocated all
>> the memory, it's simply the final allocation that pushed it over the edge.
>>
>> Here is some generic information about Solr and high heap usage:
>>
>> http://wiki.apache.org/solr/SolrPerformanceProblems#Java_Heap
>>
>> Thanks,
>> Shawn
>>
>>

Re: Java heap Space error

[Harald Kirsch]

You may want to change your solr startup script such that it creates a 
heap dump on OOM. Add -XX:+HeapDumpOnOutOfMemoryError as an option.


The heap dump can be nicely analyzed with http://www.eclipse.org/mat/.

Just increasing -Xmx is a workaround that may help to get around for a 
while. With mat you will see much clearer what is the likely cause.


Harald.

On 22.07.2014 19:37, Ameya Aware wrote:

Hi

i am running into java heap space issue. Please see below log.

ERROR - 2014-07-22 11:38:59.370; org.apache.solr.common.SolrException;
null:java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space
at
org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:790)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:439)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:368)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:636)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.OutOfMemoryError: Java heap space
at org.apache.solr.common.util.JavaBinCodec.writeStr(JavaBinCodec.java:567)
at
org.apache.solr.common.util.JavaBinCodec.writePrimitive(JavaBinCodec.java:646)
at
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:240)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:153)
at
org.apache.solr.common.util.JavaBinCodec.writeSolrInputDocument(JavaBinCodec.java:409)
at org.apache.solr.update.TransactionLog.write(TransactionLog.java:353)
at org.apache.solr.update.UpdateLog.add(UpdateLog.java:397)
at org.apache.solr.update.UpdateLog.add(UpdateLog.java:382)
at
org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:255)
at
org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:160)
at
org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69)
at
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:704)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:858)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:557)
at
org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100)
at
org.apache.solr.handler.extraction.ExtractingDocumentLoader.doAdd(ExtractingDocumentLoader.java:121)
at
org.apache.solr.handler.extraction.ExtractingDocumentLoader.addDoc(ExtractingDocumentLoader.java:126)
at
org.apache.solr.handler.extraction.ExtractingDocumentLoader.load(ExtractingDocumentLoader.java:228)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at
org.apache.solr.core.RequestHandlers

Re: Java heap space error

[Marcello Lorenzi]

Hi,
Did you set a Garbage collection strategy on your JVM ?

Marcello

On 07/24/2014 03:32 PM, Ameya Aware wrote:

Hi

I am in process of indexing around 2,00,000 documents.

I have increase java jeap space to 4 GB using below command :

java -Xmx4096M -Xms4096M -jar start.jar

Still after indexing around 15000 documents it gives java heap space error
again.


Any fix for this?

Thanks,
Ameya

Re: Java heap space error

[Ameya Aware]

I did not make any other change than this.. rest of the settings are
default.

Do i need to set garbage collection strategy?


On Thu, Jul 24, 2014 at 9:49 AM, Marcello Lorenzi 
wrote:

> Hi,
> Did you set a Garbage collection strategy on your JVM ?
>
> Marcello
>
>
> On 07/24/2014 03:32 PM, Ameya Aware wrote:
>
>> Hi
>>
>> I am in process of indexing around 2,00,000 documents.
>>
>> I have increase java jeap space to 4 GB using below command :
>>
>> java -Xmx4096M -Xms4096M -jar start.jar
>>
>> Still after indexing around 15000 documents it gives java heap space error
>> again.
>>
>>
>> Any fix for this?
>>
>> Thanks,
>> Ameya
>>
>>
>

Re: Java heap space error

[Marcello Lorenzi]

I think that on large heap is suggested to monitor the garbage 
collection behavior and try to add a strategy adapted to your 
performance.  On my production environment with a heap of 6 GB I set 
this parameter (server with 8 cores):


-server -Xms6144m -Xmx6144m -XX:MaxPermSize=512m 
-Dcom.sun.management.jmxremote -XX:+UseParNewGC -XX:+UseConcMarkSweepGC 
-XX:+CMSIncrementalMode -XX:+CMSParallelRemarkEnabled 
-XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 
-XX:ConcGCThreads=6 -XX:ParallelGCThreads=6


Marcello

On 07/24/2014 03:53 PM, Ameya Aware wrote:
I did not make any other change than this.. rest of the settings are 
default.


Do i need to set garbage collection strategy?


On Thu, Jul 24, 2014 at 9:49 AM, Marcello Lorenzi <mailto:mlore...@sorint.it>> wrote:


Hi,
Did you set a Garbage collection strategy on your JVM ?

Marcello


On 07/24/2014 03:32 PM, Ameya Aware wrote:

Hi

I am in process of indexing around 2,00,000 documents.

I have increase java jeap space to 4 GB using below command :

java -Xmx4096M -Xms4096M -jar start.jar

Still after indexing around 15000 documents it gives java heap
space error
again.


Any fix for this?

Thanks,
Ameya

Re: Java heap space error

[Ameya Aware]

ooh ok.

So you want to say that since i am using large heap but didnt set my
garbage collection, thats why i why getting java heap space error?





On Thu, Jul 24, 2014 at 9:58 AM, Marcello Lorenzi 
wrote:

>  I think that on large heap is suggested to monitor the garbage collection
> behavior and try to add a strategy adapted to your performance.  On my
> production environment with a heap of 6 GB I set this parameter (server
> with 8 cores):
>
> -server -Xms6144m -Xmx6144m -XX:MaxPermSize=512m
> -Dcom.sun.management.jmxremote -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
> -XX:+CMSIncrementalMode -XX:+CMSParallelRemarkEnabled
> -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70
> -XX:ConcGCThreads=6 -XX:ParallelGCThreads=6
>
> Marcello
>
>
> On 07/24/2014 03:53 PM, Ameya Aware wrote:
>
> I did not make any other change than this.. rest of the settings are
> default.
>
>  Do i need to set garbage collection strategy?
>
>
>  On Thu, Jul 24, 2014 at 9:49 AM, Marcello Lorenzi 
> wrote:
>
>> Hi,
>> Did you set a Garbage collection strategy on your JVM ?
>>
>> Marcello
>>
>>
>> On 07/24/2014 03:32 PM, Ameya Aware wrote:
>>
>>> Hi
>>>
>>> I am in process of indexing around 2,00,000 documents.
>>>
>>> I have increase java jeap space to 4 GB using below command :
>>>
>>> java -Xmx4096M -Xms4096M -jar start.jar
>>>
>>> Still after indexing around 15000 documents it gives java heap space
>>> error
>>> again.
>>>
>>>
>>> Any fix for this?
>>>
>>> Thanks,
>>> Ameya
>>>
>>>
>>
>
>

Re: Java heap space error

[François Schiettecatte]

A default garbage collector will be chosen for you by the VM, might help to get 
the stack trace to look at.

François

On Jul 24, 2014, at 10:06 AM, Ameya Aware  wrote:

> ooh ok.
> 
> So you want to say that since i am using large heap but didnt set my
> garbage collection, thats why i why getting java heap space error?
> 
> 
> 
> 
> 
> On Thu, Jul 24, 2014 at 9:58 AM, Marcello Lorenzi 
> wrote:
> 
>> I think that on large heap is suggested to monitor the garbage collection
>> behavior and try to add a strategy adapted to your performance.  On my
>> production environment with a heap of 6 GB I set this parameter (server
>> with 8 cores):
>> 
>> -server -Xms6144m -Xmx6144m -XX:MaxPermSize=512m
>> -Dcom.sun.management.jmxremote -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
>> -XX:+CMSIncrementalMode -XX:+CMSParallelRemarkEnabled
>> -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70
>> -XX:ConcGCThreads=6 -XX:ParallelGCThreads=6
>> 
>> Marcello
>> 
>> 
>> On 07/24/2014 03:53 PM, Ameya Aware wrote:
>> 
>> I did not make any other change than this.. rest of the settings are
>> default.
>> 
>> Do i need to set garbage collection strategy?
>> 
>> 
>> On Thu, Jul 24, 2014 at 9:49 AM, Marcello Lorenzi 
>> wrote:
>> 
>>> Hi,
>>> Did you set a Garbage collection strategy on your JVM ?
>>> 
>>> Marcello
>>> 
>>> 
>>> On 07/24/2014 03:32 PM, Ameya Aware wrote:
>>> 
>>>> Hi
>>>> 
>>>> I am in process of indexing around 2,00,000 documents.
>>>> 
>>>> I have increase java jeap space to 4 GB using below command :
>>>> 
>>>> java -Xmx4096M -Xms4096M -jar start.jar
>>>> 
>>>> Still after indexing around 15000 documents it gives java heap space
>>>> error
>>>> again.
>>>> 
>>>> 
>>>> Any fix for this?
>>>> 
>>>> Thanks,
>>>> Ameya
>>>> 
>>>> 
>>> 
>> 
>> 

Re: Java heap space error

[Boon Low]

How about simply increasing the heap size if RAM is available? You should also 
check the update handler config, e.g. auto commit, if docs aren’t being written 
to disk, they would be hanging around in memory. And “openSearcher” setting too 
as opening new searchers consumes memory, especially if expensive warm-up 
requests are configured.

Setup some graphs, monitoring the JVM heap, indexing rate, pending docs etc etc.

Boon

---
Boon Low
Lead Big Data / Search Engineer
DCT Family History


On 24 Jul 2014, at 15:12, François Schiettecatte 
mailto:fschietteca...@gmail.com>> wrote:

A default garbage collector will be chosen for you by the VM, might help to get 
the stack trace to look at.

François

On Jul 24, 2014, at 10:06 AM, Ameya Aware 
mailto:ameya.aw...@gmail.com>> wrote:

ooh ok.

So you want to say that since i am using large heap but didnt set my
garbage collection, thats why i why getting java heap space error?





On Thu, Jul 24, 2014 at 9:58 AM, Marcello Lorenzi 
mailto:mlore...@sorint.it>>
wrote:

I think that on large heap is suggested to monitor the garbage collection
behavior and try to add a strategy adapted to your performance.  On my
production environment with a heap of 6 GB I set this parameter (server
with 8 cores):

-server -Xms6144m -Xmx6144m -XX:MaxPermSize=512m
-Dcom.sun.management.jmxremote -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
-XX:+CMSIncrementalMode -XX:+CMSParallelRemarkEnabled
-XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70
-XX:ConcGCThreads=6 -XX:ParallelGCThreads=6

Marcello


On 07/24/2014 03:53 PM, Ameya Aware wrote:

I did not make any other change than this.. rest of the settings are
default.

Do i need to set garbage collection strategy?


On Thu, Jul 24, 2014 at 9:49 AM, Marcello Lorenzi 
mailto:mlore...@sorint.it>>
wrote:

Hi,
Did you set a Garbage collection strategy on your JVM ?

Marcello


On 07/24/2014 03:32 PM, Ameya Aware wrote:

Hi

I am in process of indexing around 2,00,000 documents.

I have increase java jeap space to 4 GB using below command :

java -Xmx4096M -Xms4096M -jar start.jar

Still after indexing around 15000 documents it gives java heap space
error
again.


Any fix for this?

Thanks,
Ameya








This message is confidential and may contain privileged information. You should 
not disclose its contents to any other person. If you are not the intended 
recipient, please notify the sender named above immediately. It is expressly 
declared that this e-mail does not constitute nor form part of a contract or 
unilateral obligation. Opinions, conclusions and other information in this 
message that do not relate to the official business of D.C. Thomson Family 
History shall be understood as neither given nor endorsed by it.




This email has been checked for virus and other malicious content prior to
leaving our network.

Re: Java heap space error

[Shawn Heisey]

On 7/24/2014 7:53 AM, Ameya Aware wrote:
> I did not make any other change than this.. rest of the settings are
> default.
> 
> Do i need to set garbage collection strategy?

The collector chosen and its and tuning params can have a massive impact
on performance, but it will make no difference at all if you are getting
OutOfMemoryError exceptions.  This means the program is trying to
allocate more memory than it has been told it can allocate.  Changing
the garbage collector will not change Java's response when the program
wants to allocate too much memory.

The odd location of the commas in the start of this thread make it hard
to understand exactly what numbers you were trying to say, but I think
you were saying that you were trying to index 20 documents and it
died after indexing 15000.

How big was the solr index before you started indexing, both in number
of documents and disk space consumed?  How are you doing the indexing?
Is it being done with requests to the /update handler, or are you using
the dataimport handler to import from somewhere, like a database?

Is it a single index, or distributed?  Are you running in "normal" mode
or SolrCloud?  Can you share your solrconfig.xml file so we can look for
possible problems?

I already gave you a wiki URL that gives possible reasons for needing a
very large heap, and some things you can do to reduce the requirements.

Thanks,
Shawn

Re: Java heap space error

[Steve Rowe]

On Jul 25, 2014, at 9:13 AM, Shawn Heisey  wrote:

> On 7/24/2014 7:53 AM, Ameya Aware wrote:
> The odd location of the commas in the start of this thread make it hard
> to understand exactly what numbers you were trying to say


On Jul 24, 2014, at 9:32 AM, Ameya Aware  wrote:

> I am in process of indexing around 2,00,000 documents.



1 Lakh (aka Lac) = 10^5 is written as 1,00,000 

It’s used in Bangladesh, India, Myanmar, Nepal, Pakistan, and Sri Lanka, 
roughly 1/4 of the world’s population.

RE: Java heap space error

[Toke Eskildsen]

Steve Rowe [sar...@gmail.com] wrote:
> 1 Lakh (aka Lac) = 10^5 is written as 1,00,000
>
> It’s used in Bangladesh, India, Myanmar, Nepal, Pakistan, and Sri Lanka, 
> roughly 1/4 of the world’s population.

Yet still it causes confusion and distracts from the issue. Let's just stick to 
metric, okay?

- Toke Eskildsen

SOLR OutOfMemoryError Java heap space

[Angel Tchorbadjiiski]

Hello list,

in the last couple of weeks one of my machines is experiencing 
OutOfMemoryError: Java heap space errors. In a couple of hours after 
starting the SOLR instance queries with execution times of unter 100ms 
need more than 10s to execute and many Java heap space erros appear in 
the logs. I would be grateful for any hints, how to localize/fix the 
problem.



The system is a single shard SOLR 4.6.1 instance with two cores in the 
default jetty container. The CORE1 has ~45M documents (disk size of 
32GB) with 40 fields (all stored and indexed). CORE2 has ~20M documents 
(disk size of 18GB) with 60 fields (both stored and indexed also). All 
the fields are relatively short with a maximal length of 100 characters. 
In both cores 20 of the fields are used for faceting, have a cache 
populating query on "newSearcher" event and the following cache 
configurations:


CORE1
autowarmCount="0"/>
autowarmCount="0"/>
showItems="0" />



CORE2
autowarmCount="0"/>
showItems="0" />



The OS in use is a 64bit linux with an OpenJDK 1.7 Java with 48G RAM. 
The JVM gets the following parameters to start the jetty container:


-XX:+UseCompressedOops
-XX:+UseCompressedStrings
-XX:+OptimizeStringConcat
-XX:+UseStringCache
-XX:+UseCMSInitiatingOccupancyOnly
-XX:CMSInitiatingOccupancyFraction=75
-XX:MaxTenuringThreshold=1
-XX:SurvivorRatio=8
-XX:+CMSParallelRemarkEnabled
-XX:+UseConcMarkSweepGC
-XX:+UseParNewGC


Cheers
Angel




P.S.: Here the complete error OutOfMemoryError message:

org.apache.solr.common.SolrException; null:java.lang.RuntimeException: 
java.lang.OutOfMemoryError: Java heap space
at 
org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:735)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:438)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:201)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)

at org.eclipse.jetty.server.Server.handle(Server.java:368)
at 
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at 
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at 
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
at 
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)

at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861)
at 
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
at 
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at 
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)

at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.OutOfMemoryError: Java heap space

Re: DataImporter : Java heap space

[Mani Kumar]

I am using Tomcat ...

On Mon, Apr 13, 2009 at 11:57 PM, Mani Kumar wrote:

> Hi All,
> I am trying to setup a Solr instance on my macbook.
>
> I get following errors when m trying to do a full db import ... please help
> me on this
>
> Apr 13, 2009 11:53:28 PM
> org.apache.solr.handler.dataimport.JdbcDataSource$1 call
> INFO: Creating a connection for entity slideshow with URL:
> jdbc:mysql://localhost/mydb_development
> Apr 13, 2009 11:53:29 PM
> org.apache.solr.handler.dataimport.JdbcDataSource$1 call
> INFO: Time taken for getConnection(): 319
> Apr 13, 2009 11:53:32 PM org.apache.solr.handler.dataimport.DataImporter
> doFullImport
> SEVERE: Full Import failed
> org.apache.solr.handler.dataimport.DataImportHandlerException:
> java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:400)
> at
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:221)
> at
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:164)
> at
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:312)
> at
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:370)
> at
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:351)
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at com.mysql.jdbc.Buffer.(Buffer.java:58)
> at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1444)
> at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:2840)
>
>
> My Java version
> $ java -version
> java version "1.5.0_16"
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_16-b06-284)
> Java HotSpot(TM) Client VM (build 1.5.0_16-133, mixed mode, sharing)
>
>
> Is that i need to install a new java version?
> my db is also very huge ~15 GB
>
> please do the need full ...
>
> thanks
> mani kumar
>
>

Re: DataImporter : Java heap space

[Shalin Shekhar Mangar]

On Mon, Apr 13, 2009 at 11:57 PM, Mani Kumar wrote:

> Hi All,
> I am trying to setup a Solr instance on my macbook.
>
> I get following errors when m trying to do a full db import ... please help
> me on this
>
> java.lang.OutOfMemoryError: Java heap space
>at
>
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:400)
>at
>
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:221)
>at
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:164)
>at
>
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:312)
>at
>
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:370)
>at
>
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:351)
> Caused by: java.lang.OutOfMemoryError: Java heap space
>at com.mysql.jdbc.Buffer.(Buffer.java:58)
>at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1444)
>at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:2840)
>
>
How much heap size have you allocated to the jvm?

Also see http://wiki.apache.org/solr/DataImportHandlerFaq

-- 
Regards,
Shalin Shekhar Mangar.

Re: DataImporter : Java heap space

[Mani Kumar]

Hi Shalin:

Thanks for quick response!

By defaults it was set to 1.93 MB.
But i also tried it with following command:

$  ./apache-tomcat-6.0.18/bin/startup.sh -Xmn50M -Xms300M -Xmx400M

I also tried tricks given on
http://wiki.apache.org/solr/DataImportHandlerFaq page.

what should i try next ?

Thanks!
Mani Kumar

On Tue, Apr 14, 2009 at 12:12 AM, Shalin Shekhar Mangar <
shalinman...@gmail.com> wrote:

> On Mon, Apr 13, 2009 at 11:57 PM, Mani Kumar  >wrote:
>
> > Hi All,
> > I am trying to setup a Solr instance on my macbook.
> >
> > I get following errors when m trying to do a full db import ... please
> help
> > me on this
> >
> > java.lang.OutOfMemoryError: Java heap space
> >at
> >
> >
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:400)
> >at
> >
> >
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:221)
> >at
> >
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:164)
> >at
> >
> >
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:312)
> >at
> >
> >
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:370)
> >at
> >
> >
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:351)
> > Caused by: java.lang.OutOfMemoryError: Java heap space
> >at com.mysql.jdbc.Buffer.(Buffer.java:58)
> >at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1444)
> >at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:2840)
> >
> >
> How much heap size have you allocated to the jvm?
>
> Also see http://wiki.apache.org/solr/DataImportHandlerFaq
>
> --
> Regards,
> Shalin Shekhar Mangar.
>

Re: DataImporter : Java heap space

[Ilan Rabinovitch]

Depending on your dataset and how your queries look you may very likely 
need to increase to a larger heap size.  How many queries and rows are 
required for each of your documents to be generated?


Ilan

On 4/13/09 12:21 PM, Mani Kumar wrote:

Hi Shalin:

Thanks for quick response!

By defaults it was set to 1.93 MB.
But i also tried it with following command:

$  ./apache-tomcat-6.0.18/bin/startup.sh -Xmn50M -Xms300M -Xmx400M

I also tried tricks given on
http://wiki.apache.org/solr/DataImportHandlerFaq page.

what should i try next ?

Thanks!
Mani Kumar

On Tue, Apr 14, 2009 at 12:12 AM, Shalin Shekhar Mangar<
shalinman...@gmail.com>  wrote:


On Mon, Apr 13, 2009 at 11:57 PM, Mani Kumar
wrote:



Hi All,
I am trying to setup a Solr instance on my macbook.

I get following errors when m trying to do a full db import ... please

help

me on this

java.lang.OutOfMemoryError: Java heap space
at



org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:400)

at



org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:221)

at


org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:164)

at



org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:312)

at



org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:370)

at



org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:351)

Caused by: java.lang.OutOfMemoryError: Java heap space
at com.mysql.jdbc.Buffer.(Buffer.java:58)
at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1444)
at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:2840)



How much heap size have you allocated to the jvm?

Also see http://wiki.apache.org/solr/DataImportHandlerFaq

--
Regards,
Shalin Shekhar Mangar.






--
Ilan Rabinovitch
i...@fonz.net

---
SCALE 7x: 2009 Southern California Linux Expo
Los Angeles, CA
http://www.socallinuxexpo.org

Re: DataImporter : Java heap space

[Mani Kumar]

Hi ILAN:

Only one query is required to generate a document ...
Here is my data-config.xml











and other useful info:

mysql> select * from items
+--+
| count(*) |
+--+
|   900051 |
+--+
1 row in set (0.00 sec)

Each record consist of id and title.

id  is of type int(11) and title's avg. length is 50 chars.


I am using tomcat with solr.
here is the command i am using to start it

./apache-tomcat-6.0.18/bin/startup.sh -Xmn50M -Xms300M -Xmx400M


Thanks! for help. I appreciate it.

-Mani Kumar

On Tue, Apr 14, 2009 at 2:31 AM, Ilan Rabinovitch  wrote:

> Depending on your dataset and how your queries look you may very likely
> need to increase to a larger heap size.  How many queries and rows are
> required for each of your documents to be generated?
>
> Ilan
>
>
> On 4/13/09 12:21 PM, Mani Kumar wrote:
>
>> Hi Shalin:
>>
>> Thanks for quick response!
>>
>> By defaults it was set to 1.93 MB.
>> But i also tried it with following command:
>>
>> $  ./apache-tomcat-6.0.18/bin/startup.sh -Xmn50M -Xms300M -Xmx400M
>>
>> I also tried tricks given on
>> http://wiki.apache.org/solr/DataImportHandlerFaq page.
>>
>> what should i try next ?
>>
>> Thanks!
>> Mani Kumar
>>
>> On Tue, Apr 14, 2009 at 12:12 AM, Shalin Shekhar Mangar<
>> shalinman...@gmail.com>  wrote:
>>
>>  On Mon, Apr 13, 2009 at 11:57 PM, Mani Kumar>>
>>>> wrote:
>>>>
>>>
>>>  Hi All,
>>>> I am trying to setup a Solr instance on my macbook.
>>>>
>>>> I get following errors when m trying to do a full db import ... please
>>>>
>>> help
>>>
>>>> me on this
>>>>
>>>> java.lang.OutOfMemoryError: Java heap space
>>>>at
>>>>
>>>>
>>>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:400)
>>>
>>>>at
>>>>
>>>>
>>>> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:221)
>>>
>>>>at
>>>>
>>>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:164)
>>>
>>>>at
>>>>
>>>>
>>>> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:312)
>>>
>>>>at
>>>>
>>>>
>>>> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:370)
>>>
>>>>at
>>>>
>>>>
>>>> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:351)
>>>
>>>> Caused by: java.lang.OutOfMemoryError: Java heap space
>>>>at com.mysql.jdbc.Buffer.(Buffer.java:58)
>>>>at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1444)
>>>>at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:2840)
>>>>
>>>>
>>>>  How much heap size have you allocated to the jvm?
>>>
>>> Also see http://wiki.apache.org/solr/DataImportHandlerFaq
>>>
>>> --
>>> Regards,
>>> Shalin Shekhar Mangar.
>>>
>>>
>>
>
> --
> Ilan Rabinovitch
> i...@fonz.net
>
> ---
> SCALE 7x: 2009 Southern California Linux Expo
> Los Angeles, CA
> http://www.socallinuxexpo.org
>
>

Re: DataImporter : Java heap space

[Noble Paul നോബിള്‍ नोब्ळ्]

DIH itself may not be consuming so much memory. It also includes the
memory used by Solr.

Do you have a hard limit on 400MB  , is it not possible to increase it?

On Tue, Apr 14, 2009 at 11:09 AM, Mani Kumar  wrote:
> Hi ILAN:
>
> Only one query is required to generate a document ...
> Here is my data-config.xml
>
> 
>     driver="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost/mydb_development"
> user="root" password="**" />
>    
>        
>            
>            
>        
>    
> 
>
> and other useful info:
>
> mysql> select * from items
> +--+
> | count(*) |
> +--+
> |   900051 |
> +--+
> 1 row in set (0.00 sec)
>
> Each record consist of id and title.
>
> id  is of type int(11) and title's avg. length is 50 chars.
>
>
> I am using tomcat with solr.
> here is the command i am using to start it
>
> ./apache-tomcat-6.0.18/bin/startup.sh -Xmn50M -Xms300M -Xmx400M
>
>
> Thanks! for help. I appreciate it.
>
> -Mani Kumar
>
> On Tue, Apr 14, 2009 at 2:31 AM, Ilan Rabinovitch  wrote:
>
>> Depending on your dataset and how your queries look you may very likely
>> need to increase to a larger heap size.  How many queries and rows are
>> required for each of your documents to be generated?
>>
>> Ilan
>>
>>
>> On 4/13/09 12:21 PM, Mani Kumar wrote:
>>
>>> Hi Shalin:
>>>
>>> Thanks for quick response!
>>>
>>> By defaults it was set to 1.93 MB.
>>> But i also tried it with following command:
>>>
>>> $  ./apache-tomcat-6.0.18/bin/startup.sh -Xmn50M -Xms300M -Xmx400M
>>>
>>> I also tried tricks given on
>>> http://wiki.apache.org/solr/DataImportHandlerFaq page.
>>>
>>> what should i try next ?
>>>
>>> Thanks!
>>> Mani Kumar
>>>
>>> On Tue, Apr 14, 2009 at 12:12 AM, Shalin Shekhar Mangar<
>>> shalinman...@gmail.com>  wrote:
>>>
>>>  On Mon, Apr 13, 2009 at 11:57 PM, Mani Kumar>>>
>>>>> wrote:
>>>>>
>>>>
>>>>  Hi All,
>>>>> I am trying to setup a Solr instance on my macbook.
>>>>>
>>>>> I get following errors when m trying to do a full db import ... please
>>>>>
>>>> help
>>>>
>>>>> me on this
>>>>>
>>>>> java.lang.OutOfMemoryError: Java heap space
>>>>>        at
>>>>>
>>>>>
>>>>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:400)
>>>>
>>>>>        at
>>>>>
>>>>>
>>>>> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:221)
>>>>
>>>>>        at
>>>>>
>>>>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:164)
>>>>
>>>>>        at
>>>>>
>>>>>
>>>>> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:312)
>>>>
>>>>>        at
>>>>>
>>>>>
>>>>> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:370)
>>>>
>>>>>        at
>>>>>
>>>>>
>>>>> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:351)
>>>>
>>>>> Caused by: java.lang.OutOfMemoryError: Java heap space
>>>>>        at com.mysql.jdbc.Buffer.(Buffer.java:58)
>>>>>        at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1444)
>>>>>        at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:2840)
>>>>>
>>>>>
>>>>>  How much heap size have you allocated to the jvm?
>>>>
>>>> Also see http://wiki.apache.org/solr/DataImportHandlerFaq
>>>>
>>>> --
>>>> Regards,
>>>> Shalin Shekhar Mangar.
>>>>
>>>>
>>>
>>
>> --
>> Ilan Rabinovitch
>> i...@fonz.net
>>
>> ---
>> SCALE 7x: 2009 Southern California Linux Expo
>> Los Angeles, CA
>> http://www.socallinuxexpo.org
>>
>>
>



-- 
--Noble Paul

Re: DataImporter : Java heap space

[Mani Kumar]

Here is the stack trace:

notice in stack trace *   "at
com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:1749)"*

It looks like that its trying to read whole table into memory at a time. n
thts y getting OOM.

Apr 14, 2009 11:15:01 AM org.apache.solr.handler.dataimport.DataImporter
doFullImport
SEVERE: Full Import failed
org.apache.solr.handler.dataimport.DataImportHandlerException:
java.lang.OutOfMemoryError: Java heap space
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:400)
at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:221)
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:164)
at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:312)
at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:370)
at
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:351)
Caused by: java.lang.OutOfMemoryError: Java heap space
at com.mysql.jdbc.Buffer.(Buffer.java:58)
at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1444)
at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:2840)
at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:468)
at
com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:2534)
at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:1749)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2159)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2548)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2477)
at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:741)
at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:587)
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.(JdbcDataSource.java:243)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:207)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:38)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:58)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:335)
... 5 more
Apr 14, 2009 11:15:01 AM org.apache.solr.update.DirectUpdateHandler2
rollback
INFO: start rollback
Apr 14, 2009 11:15:01 AM org.apache.solr.update.DirectUpdateHandler2
rollback
INFO: end_rollback




On Tue, Apr 14, 2009 at 11:09 AM, Mani Kumar wrote:

> Hi ILAN:
>
> Only one query is required to generate a document ...
> Here is my data-config.xml
>
> 
>  driver="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost/mydb_development"
> user="root" password="**" />
> 
> 
> 
> 
> 
> 
> 
>
> and other useful info:
>
> mysql> select * from items
> +--+
> | count(*) |
> +--+
> |   900051 |
> +--+
> 1 row in set (0.00 sec)
>
> Each record consist of id and title.
>
> id  is of type int(11) and title's avg. length is 50 chars.
>
>
> I am using tomcat with solr.
> here is the command i am using to start it
>
> ./apache-tomcat-6.0.18/bin/startup.sh -Xmn50M -Xms300M -Xmx400M
>
>
> Thanks! for help. I appreciate it.
>
> -Mani Kumar
>
> On Tue, Apr 14, 2009 at 2:31 AM, Ilan Rabinovitch  wrote:
>
>> Depending on your dataset and how your queries look you may very likely
>> need to increase to a larger heap size.  How many queries and rows are
>> required for each of your documents to be generated?
>>
>> Ilan
>>
>>
>> On 4/13/09 12:21 PM, Mani Kumar wrote:
>>
>>> Hi Shalin:
>>>
>>> Thanks for quick response!
>>>
>>> By defaults it was set to 1.93 MB.
>>> But i also tried it with following command:
>>>
>>> $  ./apache-tomcat-6.0.18/bin/startup.sh -Xmn50M -Xms300M -Xmx400M
>>>
>>> I also tried tricks given on
>>> http://wiki.apache.org/solr/DataImportHandlerFaq page.
>>>
>>> what should i try next ?
>>>
>>> Thanks!
>>> Mani Kumar
>>>
>>> On Tue, Apr 14, 2009 at 12:12 AM, Shalin Shekhar Mangar<
>>> shalinman...@gmail.com>  wrote:
>>>
>>>  On Mon, Apr 13, 2009 at 11:57 PM, Mani Kumar>>>
>>>>> wrote:
>>>>>
>>>>
>>>>  Hi All,
>>>>> I am trying to setup a Solr instance on my macbook.
>>>>>
>>>>> I get following errors when m trying to do a full db impo

Re: DataImporter : Java heap space

[Mani Kumar]

Hi Noble:


But the question is how much memory? is there any rules or something like
that? so that i can estimate the how much memory it requires?
Yeah i can increase it upto 800MB max will try it and let you know

Thanks!
Mani

2009/4/14 Noble Paul നോബിള്‍ नोब्ळ् 

> DIH itself may not be consuming so much memory. It also includes the
> memory used by Solr.
>
> Do you have a hard limit on 400MB  , is it not possible to increase it?
>
> On Tue, Apr 14, 2009 at 11:09 AM, Mani Kumar 
> wrote:
> > Hi ILAN:
> >
> > Only one query is required to generate a document ...
> > Here is my data-config.xml
> >
> > 
> > > driver="com.mysql.jdbc.Driver"
> url="jdbc:mysql://localhost/mydb_development"
> > user="root" password="**" />
> >
> >
> >
> >
> >
> >
> > 
> >
> > and other useful info:
> >
> > mysql> select * from items
> > +--+
> > | count(*) |
> > +--+
> > |   900051 |
> > +--+
> > 1 row in set (0.00 sec)
> >
> > Each record consist of id and title.
> >
> > id  is of type int(11) and title's avg. length is 50 chars.
> >
> >
> > I am using tomcat with solr.
> > here is the command i am using to start it
> >
> > ./apache-tomcat-6.0.18/bin/startup.sh -Xmn50M -Xms300M -Xmx400M
> >
> >
> > Thanks! for help. I appreciate it.
> >
> > -Mani Kumar
> >
> > On Tue, Apr 14, 2009 at 2:31 AM, Ilan Rabinovitch  wrote:
> >
> >> Depending on your dataset and how your queries look you may very likely
> >> need to increase to a larger heap size.  How many queries and rows are
> >> required for each of your documents to be generated?
> >>
> >> Ilan
> >>
> >>
> >> On 4/13/09 12:21 PM, Mani Kumar wrote:
> >>
> >>> Hi Shalin:
> >>>
> >>> Thanks for quick response!
> >>>
> >>> By defaults it was set to 1.93 MB.
> >>> But i also tried it with following command:
> >>>
> >>> $  ./apache-tomcat-6.0.18/bin/startup.sh -Xmn50M -Xms300M -Xmx400M
> >>>
> >>> I also tried tricks given on
> >>> http://wiki.apache.org/solr/DataImportHandlerFaq page.
> >>>
> >>> what should i try next ?
> >>>
> >>> Thanks!
> >>> Mani Kumar
> >>>
> >>> On Tue, Apr 14, 2009 at 12:12 AM, Shalin Shekhar Mangar<
> >>> shalinman...@gmail.com>  wrote:
> >>>
> >>>  On Mon, Apr 13, 2009 at 11:57 PM, Mani Kumar<
> manikumarchau...@gmail.com
> >>>>
> >>>>> wrote:
> >>>>>
> >>>>
> >>>>  Hi All,
> >>>>> I am trying to setup a Solr instance on my macbook.
> >>>>>
> >>>>> I get following errors when m trying to do a full db import ...
> please
> >>>>>
> >>>> help
> >>>>
> >>>>> me on this
> >>>>>
> >>>>> java.lang.OutOfMemoryError: Java heap space
> >>>>>at
> >>>>>
> >>>>>
> >>>>>
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:400)
> >>>>
> >>>>>at
> >>>>>
> >>>>>
> >>>>>
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:221)
> >>>>
> >>>>>at
> >>>>>
> >>>>>
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:164)
> >>>>
> >>>>>at
> >>>>>
> >>>>>
> >>>>>
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:312)
> >>>>
> >>>>>at
> >>>>>
> >>>>>
> >>>>>
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:370)
> >>>>
> >>>>>at
> >>>>>
> >>>>>
> >>>>>
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:351)
> >>>>
> >>>>> Caused by: java.lang.OutOfMemoryError: Java heap space
> >>>>>at com.mysql.jdbc.Buffer.(Buffer.java:58)
> >>>>>at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1444)
> >>>>>at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:2840)
> >>>>>
> >>>>>
> >>>>>  How much heap size have you allocated to the jvm?
> >>>>
> >>>> Also see http://wiki.apache.org/solr/DataImportHandlerFaq
> >>>>
> >>>> --
> >>>> Regards,
> >>>> Shalin Shekhar Mangar.
> >>>>
> >>>>
> >>>
> >>
> >> --
> >> Ilan Rabinovitch
> >> i...@fonz.net
> >>
> >> ---
> >> SCALE 7x: 2009 Southern California Linux Expo
> >> Los Angeles, CA
> >> http://www.socallinuxexpo.org
> >>
> >>
> >
>
>
>
> --
> --Noble Paul
>

Re: DataImporter : Java heap space

[Shalin Shekhar Mangar]

On Tue, Apr 14, 2009 at 11:18 AM, Mani Kumar wrote:

> Here is the stack trace:
>
> notice in stack trace *   "at
> com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:1749)"*
>
> It looks like that its trying to read whole table into memory at a time. n
> thts y getting OOM.
>
>
Mani, the data-config.xml you posted does not have the batchSize="-1"
attribute to your data source. Did you try that? This is a known bug in
MySql jdbc driver.

-- 
Regards,
Shalin Shekhar Mangar.

Re: DataImporter : Java heap space

[Mani Kumar]

Hi Shalin:
yes i tried with batchSize="-1" parameter as well

here the config i tried with




















I hope i have used batchSize parameter @ right place.


Thanks!

Mani Kumar

On Tue, Apr 14, 2009 at 11:24 AM, Shalin Shekhar Mangar <
shalinman...@gmail.com> wrote:

> On Tue, Apr 14, 2009 at 11:18 AM, Mani Kumar  >wrote:
>
> > Here is the stack trace:
> >
> > notice in stack trace *   "at
> > com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:1749)"*
> >
> > It looks like that its trying to read whole table into memory at a time.
> n
> > thts y getting OOM.
> >
> >
> Mani, the data-config.xml you posted does not have the batchSize="-1"
> attribute to your data source. Did you try that? This is a known bug in
> MySql jdbc driver.
>
> --
> Regards,
> Shalin Shekhar Mangar.
>

Re: DataImporter : Java heap space

[Shalin Shekhar Mangar]

On Tue, Apr 14, 2009 at 11:36 AM, Mani Kumar wrote:

> Hi Shalin:
> yes i tried with batchSize="-1" parameter as well
>
> here the config i tried with
>
> 
>
> driver="com.mysql.jdbc.Driver"
> url="jdbc:mysql://localhost/mydb_development"
> user="root" password="**" />
>
>
> I hope i have used batchSize parameter @ right place.
>
>
Yes that is correct. Did it still throw OOM from the same place?

I'd suggest you increase the heap and see what works for you. Also try
-server on the jvm.

-- 
Regards,
Shalin Shekhar Mangar.

Re: DataImporter : Java heap space

[Mani Kumar]

Yes its throwing the same OOM error and from same place...
yes i will try increasing the size ... just curious : how this dataimport
works?

Does it loads the whole table into memory?

Is there any estimate about how much memory it needs to create index for 1GB
of data.

thx
mani

On Tue, Apr 14, 2009 at 11:48 AM, Shalin Shekhar Mangar <
shalinman...@gmail.com> wrote:

> On Tue, Apr 14, 2009 at 11:36 AM, Mani Kumar  >wrote:
>
> > Hi Shalin:
> > yes i tried with batchSize="-1" parameter as well
> >
> > here the config i tried with
> >
> > 
> >
> > > driver="com.mysql.jdbc.Driver"
> > url="jdbc:mysql://localhost/mydb_development"
> > user="root" password="**" />
> >
> >
> > I hope i have used batchSize parameter @ right place.
> >
> >
> Yes that is correct. Did it still throw OOM from the same place?
>
> I'd suggest you increase the heap and see what works for you. Also try
> -server on the jvm.
>
> --
> Regards,
> Shalin Shekhar Mangar.
>

Re: DataImporter : Java heap space

[Noble Paul നോബിള്‍ नोब्ळ्]

DIH streams 1 row at a time.

DIH is just a component in Solr. Solr indexing also takes a lot of memory

On Tue, Apr 14, 2009 at 12:02 PM, Mani Kumar  wrote:
> Yes its throwing the same OOM error and from same place...
> yes i will try increasing the size ... just curious : how this dataimport
> works?
>
> Does it loads the whole table into memory?
>
> Is there any estimate about how much memory it needs to create index for 1GB
> of data.
>
> thx
> mani
>
> On Tue, Apr 14, 2009 at 11:48 AM, Shalin Shekhar Mangar <
> shalinman...@gmail.com> wrote:
>
>> On Tue, Apr 14, 2009 at 11:36 AM, Mani Kumar > >wrote:
>>
>> > Hi Shalin:
>> > yes i tried with batchSize="-1" parameter as well
>> >
>> > here the config i tried with
>> >
>> > 
>> >
>> >    > > driver="com.mysql.jdbc.Driver"
>> > url="jdbc:mysql://localhost/mydb_development"
>> > user="root" password="**" />
>> >
>> >
>> > I hope i have used batchSize parameter @ right place.
>> >
>> >
>> Yes that is correct. Did it still throw OOM from the same place?
>>
>> I'd suggest you increase the heap and see what works for you. Also try
>> -server on the jvm.
>>
>> --
>> Regards,
>> Shalin Shekhar Mangar.
>>
>



-- 
--Noble Paul

Re: DataImporter : Java heap space

[Bryan Talbot]

I think there is a bug in the 1.4 daily builds of data import handler  
which is causing the batchSize parameter to be ignored.  This was  
probably introduced with more recent patches to resolve variables.


The affected code is in JdbcDataSource.java

String bsz = initProps.getProperty("batchSize");
if (bsz != null) {
  bsz = (String) context.getVariableResolver().resolve(bsz);
  try {
batchSize = Integer.parseInt(bsz);
if (batchSize == -1)
  batchSize = Integer.MIN_VALUE;
  } catch (NumberFormatException e) {
LOG.warn("Invalid batch size: " + bsz);
  }
}


The call to context.getVariableResolver().resolve(bsz) is returning  
null, leading to a NumberFormatException and the batchSize never being  
set to Integer.MIN_VALUE.  MySql won't use streaming result sets in  
this case which can lead to the OOM we're seeing.



If your log file contains this entry like mine does, you're being  
affected by this bug too.


Apr 15, 2009 1:21:58 PM  
org.apache.solr.handler.dataimport.JdbcDataSource init

WARNING: Invalid batch size: null



-Bryan




On Apr 13, 2009, at Apr 13, 11:48 PM, Noble Paul നോബിള്‍  
नोब्ळ् wrote:



DIH streams 1 row at a time.

DIH is just a component in Solr. Solr indexing also takes a lot of  
memory


On Tue, Apr 14, 2009 at 12:02 PM, Mani Kumar > wrote:

Yes its throwing the same OOM error and from same place...
yes i will try increasing the size ... just curious : how this  
dataimport

works?

Does it loads the whole table into memory?

Is there any estimate about how much memory it needs to create  
index for 1GB

of data.

thx
mani

On Tue, Apr 14, 2009 at 11:48 AM, Shalin Shekhar Mangar <
shalinman...@gmail.com> wrote:


On Tue, Apr 14, 2009 at 11:36 AM, Mani Kumar 
wrote:



Hi Shalin:
yes i tried with batchSize="-1" parameter as well

here the config i tried with



   


I hope i have used batchSize parameter @ right place.



Yes that is correct. Did it still throw OOM from the same place?

I'd suggest you increase the heap and see what works for you. Also  
try

-server on the jvm.

--
Regards,
Shalin Shekhar Mangar.







--
--Noble Paul

Re: DataImporter : Java heap space

[Noble Paul നോബിള്‍ नोब्ळ्]

Hi Bryan,
Thanks a lot. It is invoking the wrong method

it should have been
bsz = context.getVariableResolver().replaceTokens(bsz);

it was a silly mistake

--Noble

On Thu, Apr 16, 2009 at 2:13 AM, Bryan Talbot  wrote:
> I think there is a bug in the 1.4 daily builds of data import handler which
> is causing the batchSize parameter to be ignored.  This was probably
> introduced with more recent patches to resolve variables.
>
> The affected code is in JdbcDataSource.java
>
>    String bsz = initProps.getProperty("batchSize");
>    if (bsz != null) {
>      bsz = (String) context.getVariableResolver().resolve(bsz);
>      try {
>        batchSize = Integer.parseInt(bsz);
>        if (batchSize == -1)
>          batchSize = Integer.MIN_VALUE;
>      } catch (NumberFormatException e) {
>        LOG.warn("Invalid batch size: " + bsz);
>      }
>    }
>
>
> The call to context.getVariableResolver().resolve(bsz) is returning null,
> leading to a NumberFormatException and the batchSize never being set to
> Integer.MIN_VALUE.  MySql won't use streaming result sets in this case which
> can lead to the OOM we're seeing.
>
>
> If your log file contains this entry like mine does, you're being affected
> by this bug too.
>
> Apr 15, 2009 1:21:58 PM org.apache.solr.handler.dataimport.JdbcDataSource
> init
> WARNING: Invalid batch size: null
>
>
>
> -Bryan
>
>
>
>
> On Apr 13, 2009, at Apr 13, 11:48 PM, Noble Paul നോബിള്‍ नोब्ळ् wrote:
>
>> DIH streams 1 row at a time.
>>
>> DIH is just a component in Solr. Solr indexing also takes a lot of memory
>>
>> On Tue, Apr 14, 2009 at 12:02 PM, Mani Kumar 
>> wrote:
>>>
>>> Yes its throwing the same OOM error and from same place...
>>> yes i will try increasing the size ... just curious : how this dataimport
>>> works?
>>>
>>> Does it loads the whole table into memory?
>>>
>>> Is there any estimate about how much memory it needs to create index for
>>> 1GB
>>> of data.
>>>
>>> thx
>>> mani
>>>
>>> On Tue, Apr 14, 2009 at 11:48 AM, Shalin Shekhar Mangar <
>>> shalinman...@gmail.com> wrote:
>>>
 On Tue, Apr 14, 2009 at 11:36 AM, Mani Kumar 
> wrote:

> Hi Shalin:
> yes i tried with batchSize="-1" parameter as well
>
> here the config i tried with
>
> 
>
>    driver="com.mysql.jdbc.Driver"
> url="jdbc:mysql://localhost/mydb_development"
> user="root" password="**" />
>
>
> I hope i have used batchSize parameter @ right place.
>
>
 Yes that is correct. Did it still throw OOM from the same place?

 I'd suggest you increase the heap and see what works for you. Also try
 -server on the jvm.

 --
 Regards,
 Shalin Shekhar Mangar.

>>>
>>
>>
>>
>> --
>> --Noble Paul
>
>



-- 
--Noble Paul

Re: DataImporter : Java heap space

[Mani Kumar]

Aah, Bryan you got it ... Thanks!
Noble: so i can hope that it'll be fixed soon :) thank you for fixing it ...
please lemme know when its done..


Thanks!
Mani Kumar
2009/4/16 Noble Paul നോബിള്‍ नोब्ळ् 

> Hi Bryan,
> Thanks a lot. It is invoking the wrong method
>
> it should have been
> bsz = context.getVariableResolver().replaceTokens(bsz);
>
> it was a silly mistake
>
> --Noble
>
> On Thu, Apr 16, 2009 at 2:13 AM, Bryan Talbot 
> wrote:
> > I think there is a bug in the 1.4 daily builds of data import handler
> which
> > is causing the batchSize parameter to be ignored.  This was probably
> > introduced with more recent patches to resolve variables.
> >
> > The affected code is in JdbcDataSource.java
> >
> >String bsz = initProps.getProperty("batchSize");
> >if (bsz != null) {
> >  bsz = (String) context.getVariableResolver().resolve(bsz);
> >  try {
> >batchSize = Integer.parseInt(bsz);
> >if (batchSize == -1)
> >  batchSize = Integer.MIN_VALUE;
> >  } catch (NumberFormatException e) {
> >LOG.warn("Invalid batch size: " + bsz);
> >  }
> >}
> >
> >
> > The call to context.getVariableResolver().resolve(bsz) is returning null,
> > leading to a NumberFormatException and the batchSize never being set to
> > Integer.MIN_VALUE.  MySql won't use streaming result sets in this case
> which
> > can lead to the OOM we're seeing.
> >
> >
> > If your log file contains this entry like mine does, you're being
> affected
> > by this bug too.
> >
> > Apr 15, 2009 1:21:58 PM org.apache.solr.handler.dataimport.JdbcDataSource
> > init
> > WARNING: Invalid batch size: null
> >
> >
> >
> > -Bryan
> >
> >
> >
> >
> > On Apr 13, 2009, at Apr 13, 11:48 PM, Noble Paul നോബിള്‍ नोब्ळ् wrote:
> >
> >> DIH streams 1 row at a time.
> >>
> >> DIH is just a component in Solr. Solr indexing also takes a lot of
> memory
> >>
> >> On Tue, Apr 14, 2009 at 12:02 PM, Mani Kumar <
> manikumarchau...@gmail.com>
> >> wrote:
> >>>
> >>> Yes its throwing the same OOM error and from same place...
> >>> yes i will try increasing the size ... just curious : how this
> dataimport
> >>> works?
> >>>
> >>> Does it loads the whole table into memory?
> >>>
> >>> Is there any estimate about how much memory it needs to create index
> for
> >>> 1GB
> >>> of data.
> >>>
> >>> thx
> >>> mani
> >>>
> >>> On Tue, Apr 14, 2009 at 11:48 AM, Shalin Shekhar Mangar <
> >>> shalinman...@gmail.com> wrote:
> >>>
>  On Tue, Apr 14, 2009 at 11:36 AM, Mani Kumar <
> manikumarchau...@gmail.com
> >
> > wrote:
> 
> > Hi Shalin:
> > yes i tried with batchSize="-1" parameter as well
> >
> > here the config i tried with
> >
> > 
> >
> >> driver="com.mysql.jdbc.Driver"
> > url="jdbc:mysql://localhost/mydb_development"
> > user="root" password="**" />
> >
> >
> > I hope i have used batchSize parameter @ right place.
> >
> >
>  Yes that is correct. Did it still throw OOM from the same place?
> 
>  I'd suggest you increase the heap and see what works for you. Also try
>  -server on the jvm.
> 
>  --
>  Regards,
>  Shalin Shekhar Mangar.
> 
> >>>
> >>
> >>
> >>
> >> --
> >> --Noble Paul
> >
> >
>
>
>
> --
> --Noble Paul
>

Re: DataImporter : Java heap space

[Shalin Shekhar Mangar]

On Thu, Apr 16, 2009 at 10:31 AM, Mani Kumar wrote:

> Aah, Bryan you got it ... Thanks!
> Noble: so i can hope that it'll be fixed soon :) thank you for fixing it
> ...
> please lemme know when its done..
>

This is fixed in trunk. The next nightly build should have this fix.

-- 
Regards,
Shalin Shekhar Mangar.

SEVERE: java.lang.OutOfMemoryError: Java heap space

[Alex Benjamen]

We're now running several solr instances on quad-cores and getting fairly good 
RPS even on 
the largest index (26MM documents) after implementing faceted queries. Things 
are looking good
except for this OutOfMemoryError which occurs every 2 hrs at peak. 
 
Note: I have browsed, searched the forums for this error and followed the most 
common advice of
increasing the memory allocation for the JVM:
/usr/bin/java  -DSTOP.PORT=8805 -DSTOP.KEY=solrstop -Xmx3584M -Xms1024M -jar 
start.jar
 
I have also reduced the autowarmcounts to 100 from 2000. Still, after running a 
couple of hrs or
so solr/ping returns 500 error with (see below)  the error in the log. We're 
running on jetty and 
the index size is 3.7Gb with 26MM documents. I don't see this error in other 
solr instances which
have an index of 1.3Gb and 8MM docs (those simply hang, which is another topic, 
but it's more rare)
 
I could allocate more physical memory, but I can't seem to increase the -Xmx 
option to 3800 I get 
an error : "Could not reserve enough space for object heap", even though I have 
more than 4Gb free.
(We're running on Intel quad core 64bit) When I try strace I'm seeing mmap2 
errors.
 
So, question which comes to mind - can this problem be solved otherwise? I'm 
sure others have used
larger indexes than this. What other settings can I tweak to get rid of this 
error... I'm currently running 
a healthcheck and simply restart the solr instance when I get a 500 error on 
the solr/ping. But this is 
ugly and bad for cache... Any ideas?

Thanks in advance!
-Alex
 
 
Jan 24, 2008 3:25:44 PM org.apache.solr.core.SolrException log
SEVERE: java.lang.OutOfMemoryError: Java heap space
at org.apache.solr.util.OpenBitSet.clone(OpenBitSet.java:539)
at org.apache.solr.search.DocSetBase.intersection(DocSet.java:201)
at 
org.apache.solr.search.SolrIndexSearcher.getDocSet(SolrIndexSearcher.java:568)
at 
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:788)
at 
org.apache.solr.search.SolrIndexSearcher.getDocList(SolrIndexSearcher.java:698)
at 
org.apache.solr.request.StandardRequestHandler.handleRequestBody(StandardRequestHandler.java:122)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:77)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:658)
at 
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:191)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:159)
at 
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
at 
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
at 
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at 
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
at 
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
at 
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
at 
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at 
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
at org.mortbay.jetty.Server.handle(Server.java:285)
at 
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
at 
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
at 
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
at 
org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)

 
 

SEVERE: java.lang.OutOfMemoryError: Java heap space

[rm_solr]

Occasionally when inserting I get the error message
 SEVERE: java.lang.OutOfMemoryError: Java heap space
Any clues how to track down when&where it's happening?
Or any good way I can get better clues how to track it down?

I'm doing inserts of about 1000 documents at a time between
commits.  Would doing a smaller number avoid this problem.
Or, better - any way I can calculate how many docs I can
insert in a batch without triggering this?

Re: java.lang.OutOfMemoryError: Java heap space

[Jörn Franke]

I think you can safely increase heap size to 1 gb or what you need.
Be aware though:
Solrs performance depends heavily on file system caches which are not on the 
heap! So you need more memory than what you configure as heap freely available. 
How much more depends on your index size. 

Another option would be to optimize the queries and maybe index, but this also 
costs time and since you do not have a really big heap.

> Am 23.07.2019 um 22:36 schrieb Mandava, Rahul :
> 
> I am using SOLR version 6.6.0 and the heap size is set to 512 MB, I believe 
> which is default. We do have almost 10 million documents in the index, we do 
> perform frequent updates (we are doing soft commit on every update: heap 
> issue was seen with and without soft commit) to the index and obviously 
> search heavily. We have experienced Heap space out of memory exception twice 
> so far in the whole year span since we started using SOLR. Since we are just 
> using default value for heap size, I am thinking to increase it and I do know 
> that high heap size can slow down the performance due to GC pauses. As we 
> can’t really come up with an ideal number that can work with any scenario, I 
> want to increase it to just 1gb only.
>  
> I did some reading around this, in which I learned that there can be lot of 
> parameters that contribute to this issue and there is no perfect way to 
> address this. And also read that increasing heap size above 2gb is where we 
> definitely be in the danger zone, since I am thinking to increase it to just 
> 1gb and if I monitor the consumption on a daily basis for a while, I should 
> be good and resolve the heap memory issue. Is that a safe assumption ??
>  
> Does anyone has experienced similar issue ?? Any thoughts or suggestions ??
>  
>  
> Below are heap usages if it helps. Usage was almost 490 mb, which makes me 
> feel with the load we have 512 mb is not enough and should be good if I 
> increase it to 1gb.
>  
>  
> 
>  
> 
>  
>  
>  
> Thanks

Re: java.lang.OutOfMemoryError: Java heap space

[Erick Erickson]

Where did you read anything about a 2G heap being “in the danger zone”? I 
routinely see heap sizes in the 16G range and greater. The default 512M is 
actually _much_ lower than it probably should be, see: 
https://issues.apache.org/jira/browse/SOLR-13446

The “danger” if you allocate too much memory is:

1> the OS needs some physical memory to hold the MMap’ed index, see: 
http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html

2> If you allocate lots more heap than you actually need, garbage collection 
can take longer.

My general rule is to _start_ with allocating no more than 50% of the available 
physical memory to Solr’s heap. In your case, since you’ve been running with 
512M 1G is perfectly reasonable, assuming you have at least 2G  physical 
memory. 

One common misconception is “if a 1G heap is good, 8G would be even better”. 
This is not true, you should only assign the heap as much memory as you need to 
avoid GC issues and OOMs. In your case, 1G is fine, 2G would also be fine 
(since GC on a 2G heap is pretty fast), but no more than that until you have 
some evidence you need it.

Best,
Erick

> On Jul 24, 2019, at 3:48 AM, Jörn Franke  wrote:
> 
> I think you can safely increase heap size to 1 gb or what you need.
> Be aware though:
> Solrs performance depends heavily on file system caches which are not on the 
> heap! So you need more memory than what you configure as heap freely 
> available. How much more depends on your index size. 
> 
> Another option would be to optimize the queries and maybe index, but this 
> also costs time and since you do not have a really big heap.
> 
>> Am 23.07.2019 um 22:36 schrieb Mandava, Rahul :
>> 
>> I am using SOLR version 6.6.0 and the heap size is set to 512 MB, I believe 
>> which is default. We do have almost 10 million documents in the index, we do 
>> perform frequent updates (we are doing soft commit on every update: heap 
>> issue was seen with and without soft commit) to the index and obviously 
>> search heavily. We have experienced Heap space out of memory exception twice 
>> so far in the whole year span since we started using SOLR. Since we are just 
>> using default value for heap size, I am thinking to increase it and I do 
>> know that high heap size can slow down the performance due to GC pauses. As 
>> we can’t really come up with an ideal number that can work with any 
>> scenario, I want to increase it to just 1gb only.
>> 
>> I did some reading around this, in which I learned that there can be lot of 
>> parameters that contribute to this issue and there is no perfect way to 
>> address this. And also read that increasing heap size above 2gb is where we 
>> definitely be in the danger zone, since I am thinking to increase it to just 
>> 1gb and if I monitor the consumption on a daily basis for a while, I should 
>> be good and resolve the heap memory issue. Is that a safe assumption ??
>> 
>> Does anyone has experienced similar issue ?? Any thoughts or suggestions ??
>> 
>> 
>> Below are heap usages if it helps. Usage was almost 490 mb, which makes me 
>> feel with the load we have 512 mb is not enough and should be good if I 
>> increase it to 1gb.
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> Thanks

RE: Solr java.lang.OutOfMemoryError: Java heap space

[Markus Jelsma]

Hi - you need to use the CursorMark feature for larger sets: 
https://cwiki.apache.org/confluence/display/solr/Pagination+of+Results
M.

 
 
-Original message-
> From:Ajinkya Kale 
> Sent: Monday 28th September 2015 20:46
> To: solr-user@lucene.apache.org; java-u...@lucene.apache.org
> Subject: Solr java.lang.OutOfMemoryError: Java heap space
> 
> Hi,
> 
> I am trying to retrieve all the documents from a solr index in a batched
> manner.
> I have 100M documents. I am retrieving them using the method proposed here
> https://nowontap.wordpress.com/2014/04/04/solr-exporting-an-index-to-an-external-file/
> I am dumping 10M document splits in each file. I get "OutOfMemoryError" if
> start is at 50M. I get the same error even if rows=10 for start=50M.
> Curl on start=0 rows=50M in one go works fine too. But things go bad when
> start is at 50M.
> My Solr version is 4.4.0.
> 
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.lucene.search.TopDocsCollector.topDocs(TopDocsCollector.java:146)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1502)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1363)
> at
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:474)
> at
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:434)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1904)
> 
> --aj
> 

Re: Solr java.lang.OutOfMemoryError: Java heap space

[Ajinkya Kale]

If I am not wrong this works only with Solr version > 4.7.0 ?
On Mon, Sep 28, 2015 at 12:23 PM Markus Jelsma 
wrote:

> Hi - you need to use the CursorMark feature for larger sets:
> https://cwiki.apache.org/confluence/display/solr/Pagination+of+Results
> M.
>
>
>
> -Original message-
> > From:Ajinkya Kale 
> > Sent: Monday 28th September 2015 20:46
> > To: solr-user@lucene.apache.org; java-u...@lucene.apache.org
> > Subject: Solr java.lang.OutOfMemoryError: Java heap space
> >
> > Hi,
> >
> > I am trying to retrieve all the documents from a solr index in a batched
> > manner.
> > I have 100M documents. I am retrieving them using the method proposed
> here
> >
> https://nowontap.wordpress.com/2014/04/04/solr-exporting-an-index-to-an-external-file/
> > I am dumping 10M document splits in each file. I get "OutOfMemoryError"
> if
> > start is at 50M. I get the same error even if rows=10 for start=50M.
> > Curl on start=0 rows=50M in one go works fine too. But things go bad when
> > start is at 50M.
> > My Solr version is 4.4.0.
> >
> > Caused by: java.lang.OutOfMemoryError: Java heap space
> > at
> >
> org.apache.lucene.search.TopDocsCollector.topDocs(TopDocsCollector.java:146)
> > at
> >
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1502)
> > at
> >
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1363)
> > at
> >
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:474)
> > at
> >
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:434)
> > at
> >
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208)
> > at
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1904)
> >
> > --aj
> >
>

Re: Solr java.lang.OutOfMemoryError: Java heap space

[Gili Nachum]

If you can't use CursorMark, then I suggest not using the start parameter,
instead sort asc by a unique field and and range the query to records with
a field value larger then the last doc you read. Then set rows to be
whatever you found can fit in memory.

On Mon, Sep 28, 2015 at 10:59 PM, Ajinkya Kale 
wrote:

> If I am not wrong this works only with Solr version > 4.7.0 ?
> On Mon, Sep 28, 2015 at 12:23 PM Markus Jelsma  >
> wrote:
>
> > Hi - you need to use the CursorMark feature for larger sets:
> > https://cwiki.apache.org/confluence/display/solr/Pagination+of+Results
> > M.
> >
> >
> >
> > -Original message-
> > > From:Ajinkya Kale 
> > > Sent: Monday 28th September 2015 20:46
> > > To: solr-user@lucene.apache.org; java-u...@lucene.apache.org
> > > Subject: Solr java.lang.OutOfMemoryError: Java heap space
> > >
> > > Hi,
> > >
> > > I am trying to retrieve all the documents from a solr index in a
> batched
> > > manner.
> > > I have 100M documents. I am retrieving them using the method proposed
> > here
> > >
> >
> https://nowontap.wordpress.com/2014/04/04/solr-exporting-an-index-to-an-external-file/
> > > I am dumping 10M document splits in each file. I get "OutOfMemoryError"
> > if
> > > start is at 50M. I get the same error even if rows=10 for start=50M.
> > > Curl on start=0 rows=50M in one go works fine too. But things go bad
> when
> > > start is at 50M.
> > > My Solr version is 4.4.0.
> > >
> > > Caused by: java.lang.OutOfMemoryError: Java heap space
> > > at
> > >
> >
> org.apache.lucene.search.TopDocsCollector.topDocs(TopDocsCollector.java:146)
> > > at
> > >
> >
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1502)
> > > at
> > >
> >
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1363)
> > > at
> > >
> >
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:474)
> > > at
> > >
> >
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:434)
> > > at
> > >
> >
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208)
> > > at
> > >
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> > > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1904)
> > >
> > > --aj
> > >
> >
>

RE: Solr java.lang.OutOfMemoryError: Java heap space

[will martin]

http://opensourceconnections.com/blog/2014/07/13/reindexing-collections-with-solrs-cursor-support/



-Original Message-
From: Ajinkya Kale [mailto:kaleajin...@gmail.com] 
Sent: Monday, September 28, 2015 2:46 PM
To: solr-user@lucene.apache.org; java-u...@lucene.apache.org
Subject: Solr java.lang.OutOfMemoryError: Java heap space

Hi,

I am trying to retrieve all the documents from a solr index in a batched manner.
I have 100M documents. I am retrieving them using the method proposed here 
https://nowontap.wordpress.com/2014/04/04/solr-exporting-an-index-to-an-external-file/
I am dumping 10M document splits in each file. I get "OutOfMemoryError" if 
start is at 50M. I get the same error even if rows=10 for start=50M.
Curl on start=0 rows=50M in one go works fine too. But things go bad when start 
is at 50M.
My Solr version is 4.4.0.

Caused by: java.lang.OutOfMemoryError: Java heap space at
org.apache.lucene.search.TopDocsCollector.topDocs(TopDocsCollector.java:146)
at
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1502)
at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1363)
at
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:474)
at
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:434)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1904)

--aj

Java heap space exception in 4.2.1

[J Mohamed Zahoor]

Hi 

I moved to 4.2.1 from 4.1 recently.. everything was working fine until i added 
few more stats query..
Now i am getting this error frequently that solr does not run even for 2 
minutes continuously.
All 5GB is getting used instantaneously in few queries...


SEVERE: null:java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap 
space
at 
org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:653)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:366)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:560)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1072)
at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1006)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:365)
at 
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:485)
at 
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at 
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:937)
at 
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:998)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:856)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
at 
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at 
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:722)



one thing i did was when we moved from 4.1 to 4.2.1 i changed only the solr.war 
and left other jars and config as it is...
WIll that create a problem...??

./zahoor

RE: SOLR OutOfMemoryError Java heap space

[Toke Eskildsen]

Angel Tchorbadjiiski [angel.tchorbadjii...@antibodies-online.com] wrote:

[Single shard / 2 cores Solr 4.6.1, 65M docs / 50GB, 20 facet fields]

> The OS in use is a 64bit linux with an OpenJDK 1.7 Java with 48G RAM.

I did not see your memory allocation anywhere. What is your Xmx?

> P.S.: Here the complete error OutOfMemoryError message:

> org.apache.solr.common.SolrException; null:java.lang.RuntimeException:
> java.lang.OutOfMemoryError: Java heap space
>  at
> org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:735)
...
>  at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
>  at java.lang.Thread.run(Thread.java:724)
> Caused by: java.lang.OutOfMemoryError: Java heap space

A quick sanity check: Is your max user processes higher than 1024?
(run 'ulimit -a' to inspect it)


- Toke Eskildsen, State and University Library, Denmark

Re: SOLR OutOfMemoryError Java heap space

[Shawn Heisey]

On 3/4/2014 2:23 AM, Angel Tchorbadjiiski wrote:
in the last couple of weeks one of my machines is experiencing 
OutOfMemoryError: Java heap space errors. In a couple of hours after 
starting the SOLR instance queries with execution times of unter 100ms 
need more than 10s to execute and many Java heap space erros appear in 
the logs. I would be grateful for any hints, how to localize/fix the 
problem.



The system is a single shard SOLR 4.6.1 instance with two cores in the 
default jetty container. The CORE1 has ~45M documents (disk size of 
32GB) with 40 fields (all stored and indexed). CORE2 has ~20M 
documents (disk size of 18GB) with 60 fields (both stored and indexed 
also). All the fields are relatively short with a maximal length of 
100 characters. In both cores 20 of the fields are used for faceting, 
have a cache populating query on "newSearcher" event and the following 
cache configurations:


It may be your facets that are killing you here.  As Toke mentioned, you 
have not indicated what your max heap is.20 separate facet fields with 
millions of documents will use a lot of fieldcache memory if you use the 
standard facet.method, fc.


Try adding facet.method=enum to all your facet queries, or you can put 
it in the defaults section of each request handler definition.  
Alternatively, you can add docValues to all your facet fields, but that 
will require a full reindex.


http://wiki.apache.org/solr/SolrPerformanceProblems#Java_Heap

Thanks,
Shawn

Re: SOLR OutOfMemoryError Java heap space

[Angel Tchorbadjiiski]

Hi Toke,

thank you for the mail.

On 04.03.2014 11:20, Toke Eskildsen wrote:

Angel Tchorbadjiiski [angel.tchorbadjii...@antibodies-online.com] wrote:

[Single shard / 2 cores Solr 4.6.1, 65M docs / 50GB, 20 facet fields]


The OS in use is a 64bit linux with an OpenJDK 1.7 Java with 48G RAM.


I did not see your memory allocation anywhere. What is your Xmx?
At the moment I dont use it. The instance allocates 12G without the 
parameter set.




P.S.: Here the complete error OutOfMemoryError message:



org.apache.solr.common.SolrException; null:java.lang.RuntimeException:
java.lang.OutOfMemoryError: Java heap space
  at
org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:735)

...

  at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
  at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.OutOfMemoryError: Java heap space


A quick sanity check: Is your max user processes higher than 1024?
(run 'ulimit -a' to inspect it)

max user processes  (-u) 387083

It is set high enough.

Cheers
Angel

Re: SOLR OutOfMemoryError Java heap space

[Angel Tchorbadjiiski]

Hi Shawn,


It may be your facets that are killing you here.  As Toke mentioned, you
have not indicated what your max heap is.20 separate facet fields with
millions of documents will use a lot of fieldcache memory if you use the
standard facet.method, fc.

Try adding facet.method=enum to all your facet queries, or you can put
it in the defaults section of each request handler definition.

Ok, that is easy to try out.


Alternatively, you can add docValues to all your facet fields, but that
will require a full reindex.
This will not be so easy to handle, so I'll try it only if nothing else 
helps.


Thanks a lot,
Angel

Re: SOLR OutOfMemoryError Java heap space

[Toke Eskildsen]

On Wed, 2014-03-05 at 09:59 +0100, Angel Tchorbadjiiski wrote:
> On 04.03.2014 11:20, Toke Eskildsen wrote:
> > Angel Tchorbadjiiski [angel.tchorbadjii...@antibodies-online.com] wrote:
> >
> > [Single shard / 2 cores Solr 4.6.1, 65M docs / 50GB, 20 facet fields]
> >
> >> The OS in use is a 64bit linux with an OpenJDK 1.7 Java with 48G RAM.
> >
> > I did not see your memory allocation anywhere. What is your Xmx?
> At the moment I dont use it. The instance allocates 12G without the 
> parameter set.

I have little experience with OpenJDK, but a quick search suggests that
the default Xmx is physical memory/4 which is indeed 12G for your
machine. I strongly recommend that you set Xmx explicitly instead as the
value should be tuned to your concrete Solr deploy and not whatever
amount of RAM your machine happens to have.

Shawn suggests facets to be the culprit and I find it a fair suggestion.
The stack trace you pasted did not state exactly what caused the OOM -
was is a complete trace? You might find better information in the Solr
log. If it is facet related, it is probably during uninversion.


A gotcha in Solr faceting is that it is quite often the number of
documents, rather than the number of facets or facet values, that
requires a lot of memory.

Extremely loose numbers: Field-faceting on 65M documents (wildly
guessing 5000 unique values and 2 references/doc) is somewhere around
65M*log2(65M*2) + 2*65M*log(5000) bits ~= 65M*28 + 130M*13 ~= 400MB.

As Solr does not come with facet structure collapsing, each facet is
independent of the others, so with the above estimate, 8GB will be used
for faceting.

Before DocValues became the answer, I wrote a bit about it here:
http://sbdevel.wordpress.com/2013/04/16/you-are-faceting-itwrong/

- Toke Eskildsen, State and University Library, Denmark

Re: SOLR OutOfMemoryError Java heap space

[Angel Tchorbadjiiski]

On 05.03.2014 11:51, Toke Eskildsen wrote:

On Wed, 2014-03-05 at 09:59 +0100, Angel Tchorbadjiiski wrote:

On 04.03.2014 11:20, Toke Eskildsen wrote:

Angel Tchorbadjiiski [angel.tchorbadjii...@antibodies-online.com] wrote:

[Single shard / 2 cores Solr 4.6.1, 65M docs / 50GB, 20 facet fields]


The OS in use is a 64bit linux with an OpenJDK 1.7 Java with 48G RAM.


I did not see your memory allocation anywhere. What is your Xmx?

At the moment I dont use it. The instance allocates 12G without the
parameter set.


I have little experience with OpenJDK, but a quick search suggests that
the default Xmx is physical memory/4 which is indeed 12G for your
machine. I strongly recommend that you set Xmx explicitly instead as the
value should be tuned to your concrete Solr deploy and not whatever
amount of RAM your machine happens to have.

Yes, that is right. I've now both -Xms and -Xmx set.


Shawn suggests facets to be the culprit and I find it a fair suggestion.
As both cores are used primarily for faceting, this sounds very 
reasonable:).



The stack trace you pasted did not state exactly what caused the OOM -
was is a complete trace? You might find better information in the Solr
log. If it is facet related, it is probably during uninversion.
The traces were mostly exactly the same as the included one, but I'll 
have a closer look at the logs again.




A gotcha in Solr faceting is that it is quite often the number of
documents, rather than the number of facets or facet values, that
requires a lot of memory.

Extremely loose numbers: Field-faceting on 65M documents (wildly
guessing 5000 unique values and 2 references/doc) is somewhere around
65M*log2(65M*2) + 2*65M*log(5000) bits ~= 65M*28 + 130M*13 ~= 400MB.

As Solr does not come with facet structure collapsing, each facet is
independent of the others, so with the above estimate, 8GB will be used
for faceting.


Thank you for the example:-).

I had a look at the inverted index in Solr and the problems resulting 
form it in respect to memory needed. It seems, that the docValue fields 
will be the solution for the problem.




Before DocValues became the answer, I wrote a bit about it here:
http://sbdevel.wordpress.com/2013/04/16/you-are-faceting-itwrong/


I'll have a look at the post, thank you very much.

Cheers
Angel

Re: SOLR OutOfMemoryError Java heap space

[Angel Tchorbadjiiski]

Hi Shawn,

On 05.03.2014 10:05, Angel Tchorbadjiiski wrote:

Hi Shawn,


It may be your facets that are killing you here.  As Toke mentioned, you
have not indicated what your max heap is.20 separate facet fields with
millions of documents will use a lot of fieldcache memory if you use the
standard facet.method, fc.

Try adding facet.method=enum to all your facet queries, or you can put
it in the defaults section of each request handler definition.

Ok, that is easy to try out.

Changing the facet.method does not help really as the performance of the 
queries is really bad. This lies mostly on the small cache values, but 
even trying to tune them for the "enum" case didn't help much.


The number of documents and unique facet values seems to be too high. 
Trying to cache them even with a size of 512 results in many misses and 
Solr tries to repopulate the cache all the time. This makes the 
performances even worse.


Cheers
Angel

Re: SOLR OutOfMemoryError Java heap space

[Shawn Heisey]

On 3/5/2014 4:40 AM, Angel Tchorbadjiiski wrote:
> Hi Shawn,
> 
> On 05.03.2014 10:05, Angel Tchorbadjiiski wrote:
>> Hi Shawn,
>>
>>> It may be your facets that are killing you here.  As Toke mentioned, you
>>> have not indicated what your max heap is.20 separate facet fields with
>>> millions of documents will use a lot of fieldcache memory if you use the
>>> standard facet.method, fc.
>>>
>>> Try adding facet.method=enum to all your facet queries, or you can put
>>> it in the defaults section of each request handler definition.
>> Ok, that is easy to try out.
>>
> Changing the facet.method does not help really as the performance of the
> queries is really bad. This lies mostly on the small cache values, but
> even trying to tune them for the "enum" case didn't help much.
> 
> The number of documents and unique facet values seems to be too high.
> Trying to cache them even with a size of 512 results in many misses and
> Solr tries to repopulate the cache all the time. This makes the
> performances even worse.

Good performance with Solr requires a fair amount of memory.  You have
two choices when it comes to where that memory gets used - inside Solr
in the form of caches, or free memory, available to the operating system
for caching purposes.

Solr caches are really amazing things.  Data gathered for one query can
significantly speed up another query, because part (or all) of that
query can be simply skipped, the results read right out of the cache.

There are two potential problems with relying exclusively on Solr
caches, though.  One is that they require Java heap memory, which
requires garbage collection.  A large heap causes GC issues, some of
which can be alleviated by GC tuning.  The other problem is that you
must actually do a query in order to get the data into the cache.  WHen
you do a commit and open a new searcher, that cache data does away, so
you have to do the query over again.

The primary reason for slow uncached queries is disk access.  Reading
index data off the disk is a glacial process, comparatively speaking.
This is where OS disk caching becomes a benefit.  Most queries, even
complex ones, become lightning fast if all of the relevant index data is
already in RAM and no disk access is required.  When queries are fast to
begin with, you can reduce the cache sizes in Solr, reducing the heap
requirements.  With a smaller heap, more memory is available for the OS
disk cache.

The facet.method=enum parameter shifts the RAM requirement from Solr to
the OS.  It does not really reduce the amount of required system memory.
 Because disk caching is a kernel level feature and does not utilize
garbage collection, it is far more efficient than Solr ever could be at
caching *raw* data.  Solr's caches are designed for *processed* data.

What this all boils down to is that I suspect you'll simply need more
memory on the machine.  With facets on so many fields, your queries are
probably touching nearly the entire index, so you'll want to put the
entire index into RAM.

Therefore, after Solr allocates its heap and any other programs on the
system allocate their required memory, you must have enough left memory
over to fit all (or most) of your 50GB index data.  Combine this with
facet.method=enum and everything should be good.

Thanks,
Shawn

Re: SOLR OutOfMemoryError Java heap space

[Angel Tchorbadjiiski]

Hi Shawn,

a big thanks for the long and detailed answer. I am aware of how linux 
uses free RAM for caching and the the problems related to jvm and GC. It 
is nice to hear how this correlates to Solr. I'll take some time and 
think over it. The facet.method=enum and probably a combination of 
DocValue-Fields could be the solution needed in this case.


Thanks again to both of you and Toke for the feedback!

Cheers
Angel

On 05.03.2014 17:06, Shawn Heisey wrote:

On 3/5/2014 4:40 AM, Angel Tchorbadjiiski wrote:

Hi Shawn,

On 05.03.2014 10:05, Angel Tchorbadjiiski wrote:

Hi Shawn,


It may be your facets that are killing you here.  As Toke mentioned, you
have not indicated what your max heap is.20 separate facet fields with
millions of documents will use a lot of fieldcache memory if you use the
standard facet.method, fc.

Try adding facet.method=enum to all your facet queries, or you can put
it in the defaults section of each request handler definition.

Ok, that is easy to try out.


Changing the facet.method does not help really as the performance of the
queries is really bad. This lies mostly on the small cache values, but
even trying to tune them for the "enum" case didn't help much.

The number of documents and unique facet values seems to be too high.
Trying to cache them even with a size of 512 results in many misses and
Solr tries to repopulate the cache all the time. This makes the
performances even worse.


Good performance with Solr requires a fair amount of memory.  You have
two choices when it comes to where that memory gets used - inside Solr
in the form of caches, or free memory, available to the operating system
for caching purposes.

Solr caches are really amazing things.  Data gathered for one query can
significantly speed up another query, because part (or all) of that
query can be simply skipped, the results read right out of the cache.

There are two potential problems with relying exclusively on Solr
caches, though.  One is that they require Java heap memory, which
requires garbage collection.  A large heap causes GC issues, some of
which can be alleviated by GC tuning.  The other problem is that you
must actually do a query in order to get the data into the cache.  WHen
you do a commit and open a new searcher, that cache data does away, so
you have to do the query over again.

The primary reason for slow uncached queries is disk access.  Reading
index data off the disk is a glacial process, comparatively speaking.
This is where OS disk caching becomes a benefit.  Most queries, even
complex ones, become lightning fast if all of the relevant index data is
already in RAM and no disk access is required.  When queries are fast to
begin with, you can reduce the cache sizes in Solr, reducing the heap
requirements.  With a smaller heap, more memory is available for the OS
disk cache.

The facet.method=enum parameter shifts the RAM requirement from Solr to
the OS.  It does not really reduce the amount of required system memory.
  Because disk caching is a kernel level feature and does not utilize
garbage collection, it is far more efficient than Solr ever could be at
caching *raw* data.  Solr's caches are designed for *processed* data.

What this all boils down to is that I suspect you'll simply need more
memory on the machine.  With facets on so many fields, your queries are
probably touching nearly the entire index, so you'll want to put the
entire index into RAM.

Therefore, after Solr allocates its heap and any other programs on the
system allocate their required memory, you must have enough left memory
over to fit all (or most) of your 50GB index data.  Combine this with
facet.method=enum and everything should be good.

Re: SOLR OutOfMemoryError Java heap space

[Divyang Shah]

hi,
heap problem is due to memory full.
you should remove unnecessary data and restart server once.



On Thursday, 6 March 2014 10:39 AM, Angel Tchorbadjiiski 
 wrote:
 
Hi Shawn,

a big thanks for the long and detailed answer. I am aware of how linux 
uses free RAM for caching and the the problems related to jvm and GC. It 
is nice to hear how this correlates to Solr. I'll take some time and 
think over it. The facet.method=enum and probably a combination of 
DocValue-Fields could be the solution needed in this case.

Thanks again to both of you and Toke for the feedback!

Cheers
Angel


On 05.03.2014 17:06, Shawn Heisey wrote:
> On 3/5/2014 4:40 AM, Angel Tchorbadjiiski wrote:
>> Hi Shawn,
>>
>> On 05.03.2014 10:05, Angel Tchorbadjiiski wrote:
>>> Hi Shawn,
>>>
 It may be your facets that are killing you here.  As Toke mentioned, you
 have not indicated what your max heap is.20 separate facet fields with
 millions of documents will use a lot of fieldcache memory if you use the
 standard facet.method, fc.

 Try adding facet.method=enum to all your facet queries, or you can put
 it in the defaults section of each request handler definition.
>>> Ok, that is easy to try out.
>>>
>> Changing the facet.method does not help really as the performance of the
>> queries is really bad. This lies mostly on the small cache values, but
>> even trying to tune them for the "enum" case didn't help much.
>>
>> The number of documents and unique facet values seems to be too high.
>> Trying to cache them even with a size of 512 results in many misses and
>> Solr tries to repopulate the cache all the time. This makes the
>> performances even worse.
>
> Good performance with Solr requires a fair amount of memory.  You have
> two choices when it comes to where that memory gets used - inside Solr
> in the form of caches, or free memory, available to the operating system
> for caching purposes.
>
> Solr caches are really amazing things.  Data gathered for one query can
> significantly speed up another query, because part (or all) of that
> query can be simply skipped, the results read right out of the cache.
>
> There are two potential problems with relying exclusively on Solr
> caches, though.  One is that they require Java heap memory, which
> requires garbage collection.  A large heap causes GC issues, some of
> which can be alleviated by GC tuning.  The other problem is that you
> must actually do a query in order to get the data into the cache.  WHen
> you do a commit and open a new searcher, that cache data does away, so
> you have to do the query over again.
>
> The primary reason for slow uncached queries is disk access.  Reading
> index data off the disk is a glacial process, comparatively speaking.
> This is where OS disk caching becomes a benefit.  Most queries, even
> complex ones, become lightning fast if all of the relevant index data is
> already in RAM and no disk access is required.  When queries are fast to
> begin with, you can reduce the cache sizes in Solr, reducing the heap
> requirements.  With a smaller heap, more memory is available for the OS
> disk cache.
>
> The facet.method=enum parameter shifts the RAM requirement from Solr to
> the OS.  It does not really reduce the amount of required system memory.
>   Because disk caching is a kernel level feature and does not utilize
> garbage collection, it is far more efficient than Solr ever could be at
> caching *raw* data.  Solr's caches are designed for *processed* data.
>
> What this all boils down to is that I suspect you'll simply need more
> memory on the machine.  With facets on so many fields, your queries are
> probably touching nearly the entire index, so you'll want to put the
> entire index into RAM.
>
> Therefore, after Solr allocates its heap and any other programs on the
> system allocate their required memory, you must have enough left memory
> over to fit all (or most) of your 50GB index data.  Combine this with
> facet.method=enum and everything should be good.

java heap space error when faceting

[Matt Mitchell]

I have an index with more than 6 million docs. All is well, until I turn on
faceting and specify a facet.field. There is only about unique 20 values for
this particular facet throughout the entire index. I was able to make things
a little better by using facet.method=enum. That seems to work, until I add
another facet.field to the request, which is another facet that doesn't have
that many unique values. I utlimately end up running out of heap space
memory. I should also mention that in every case, the "rows" param is set to
0.

I've thrown as much memory as I can at the JVM (+3G for start-up and max),
tweaked filter cache settings etc.. I can't seem to get this error to go
away. Anyone have any tips to throw my way?

-- using a recent nighlty build of solr 1.5 dev and Jetty as my servlet
container.

Thanks!
Matt

Re: SEVERE: java.lang.OutOfMemoryError: Java heap space

[Brian Whitman]

On Jan 28, 2008, at 7:06 PM, Leonardo Santagada wrote:


On 28/01/2008, at 20:44, Alex Benjamen wrote:




I could allocate more physical memory, but I can't seem to increase  
the -Xmx option to 3800 I get
an error : "Could not reserve enough space for object heap", even  
though I have more than 4Gb free.
(We're running on Intel quad core 64bit) When I try strace I'm  
seeing mmap2 errors.

]

I don't know much about java... but can you get any program to map  
more than 4gb of memory? I know windows has hard limits on how much  
memory you can map to one process and linux I think has some limit  
too. Of course it can be configured but maybe it is just a system  
configuration problem.


We use 10GB of ram in one of our solr installs. You need to make sure  
your java is 64 bit though.  Alex, what does your java -version show?  
Mine shows


java version "1.6.0_03"
Java(TM) SE Runtime Environment (build 1.6.0_03-b05)
Java HotSpot(TM) 64-Bit Server VM (build 1.6.0_03-b05, mixed mode)

And I run it with -mx1m -ms5000m

Re: SEVERE: java.lang.OutOfMemoryError: Java heap space

[Leonardo Santagada]

On 28/01/2008, at 20:44, Alex Benjamen wrote:
Note: I have browsed, searched the forums for this error and  
followed the most common advice of

increasing the memory allocation for the JVM:
/usr/bin/java  -DSTOP.PORT=8805 -DSTOP.KEY=solrstop -Xmx3584M - 
Xms1024M -jar start.jar



[snip]


I could allocate more physical memory, but I can't seem to increase  
the -Xmx option to 3800 I get
an error : "Could not reserve enough space for object heap", even  
though I have more than 4Gb free.
(We're running on Intel quad core 64bit) When I try strace I'm  
seeing mmap2 errors.



I don't know much about java... but can you get any program to map  
more than 4gb of memory? I know windows has hard limits on how much  
memory you can map to one process and linux I think has some limit  
too. Of course it can be configured but maybe it is just a system  
configuration problem.


But still there should be a way to tell jetty to only have a certain  
amount of threads/process/requests at a time, so you never risk the  
problem of getting out of memory problems.


--
Leonardo Santagada

RE: SEVERE: java.lang.OutOfMemoryError: Java heap space

[Alex Benjamen]

>We use 10GB of ram in one of our solr installs. You need to make sure 
>your java is 64 bit though.  Alex, what does your java -version show? 
>Mine shows

>java version "1.6.0_03"
>Java(TM) SE Runtime Environment (build 1.6.0_03-b05)
>Java HotSpot(TM) 64-Bit Server VM (build 1.6.0_03-b05, mixed mode)

>And I run it with -mx1m -ms5000m

Brian, good point - I'm running on both AMD and Intel platform, and on AMD 
it does say :
$ java -version
java version "1.6.0_10-ea"
Java(TM) SE Runtime Environment (build 1.6.0_10-ea-b07)
Java HotSpot(TM) 64-Bit Server VM (build 1.6.0_10-ea-b07, mixed mode)
 
But on Intel, where I'm having the problem it  shows:
java version "1.6.0_10-ea"
Java(TM) SE Runtime Environment (build 1.6.0_10-ea-b10)
Java HotSpot(TM) Server VM (build 11.0-b09, mixed mode)
 
I can't seem to find the Intel 64 bit JDK binary, can you pls. send me the link?
I was downloading from here:
http://download.java.net/jdk6/
 
Thanks!
-Alex

Re: SEVERE: java.lang.OutOfMemoryError: Java heap space

[Brian Whitman]

But on Intel, where I'm having the problem it  shows:
java version "1.6.0_10-ea"
Java(TM) SE Runtime Environment (build 1.6.0_10-ea-b10)
Java HotSpot(TM) Server VM (build 11.0-b09, mixed mode)

I can't seem to find the Intel 64 bit JDK binary, can you pls. send  
me the link?

I was downloading from here:
http://download.java.net/jdk6/




Install the AMD64 version. (Confusingly, AMD64 is a spec name for  
EM64T, which is now what both AMD and Intel use)
If that still doesn't work, is it possible that your machine/kernel is  
not set up to support 64 bit?

RE: SEVERE: java.lang.OutOfMemoryError: Java heap space

[Alex Benjamen]

>Install the AMD64 version. (Confusingly, AMD64 is a spec name for 
>EM64T, which is now what both AMD and Intel use)
>If that still doesn't work, is it possible that your machine/kernel is 
>not set up to support 64 bit?
 
I was confused by the naming convention. Seems to work fine now, well,
I mean fine that it can be started with a larger heap. We'll see how it runs 
overnight

Thanks again,
Alex

RE: SEVERE: java.lang.OutOfMemoryError: Java heap space

[Alex Benjamen]

Thanks to all who responded. Things are running well! The IBM version 
of the JRE for Intel 64 seems to run good, and the stalling issue has 
dissappeared.
(when the solr instance stops responding and freezes up)

What I learned is that solr is a great product but needs "tuning" to fit the 
usage.
Changing some config settings can make the difference between a usable and 
unusable system. 
 
I guess for most smaller indeces things will run fine out of the box, but if you
have a large index then you have to play around a bit

-Alex

>Install the AMD64 version. (Confusingly, AMD64 is a spec name for
>EM64T, which is now what both AMD and Intel use)
>If that still doesn't work, is it possible that your machine/kernel is
>not set up to support 64 bit?

I was confused by the naming convention. Seems to work fine now, well,
I mean fine that it can be started with a larger heap. We'll see how it runs 
overnight

Thanks again,
Alex

Re: SEVERE: java.lang.OutOfMemoryError: Java heap space

[Yonik Seeley]

On 3/22/06, [EMAIL PROTECTED]
<[EMAIL PROTECTED]> wrote:
>
> Occasionally when inserting I get the error message
>   SEVERE: java.lang.OutOfMemoryError: Java heap space
> Any clues how to track down when&where it's happening?
> Or any good way I can get better clues how to track it down?

What's the heap size of the JVM.  The default is normally pretty small.
If you are using the example, just put it on the command line...

java -Xmx256m -jar start.jar

> I'm doing inserts of about 1000 documents at a time between
> commits.  Would doing a smaller number avoid this problem.

No, that shouldn't be a problem.
Solr only keeps a single hashtable entry for each unique id of
uncommitted documents so it can delete duplicates during the commit.

The exception probably hits when lucene needs to merge segments. 
There isn't much you can do to avoid that memory usage except increase
the heap size.
BTW, How many indexed fields do you have?

-Yonik

Getting OutOfMemoryError: Java heap space in Solr

[yuvaraj ponnuswamy]

Hi,

I am getting the OutofMemory Error: "java.lang.OutOfMemoryError: Java heap 
space" often in production due to the particular Treemap is taking more memory 
in the JVM.

When i looked into the config files I am having the entity called 
UserQryDocument where i am fetching the data from certain tables.
Again i have a sub entiry called "UserLocation" where i am using the 
CachedSqlEntityProcessor to get the fields from Cache. It seems like it has the 
total of 2,00,000 records total.
processor="CachedSqlEntityProcessor" cacheKey="user_pin" 
cacheLookup="UserQueryDocumentNonAuthor.DocKey">

Like this i have some other different entity and there also i am using this 
CachedSqlEntityProcessor in the sub entity.

But when i looked into the Heap Dump : java_pid57.hprof i am able to see the 
TreeMap is causing the problem.

But not able to find which entity is causing this issue.I am using the IBM Heap 
Ananlyser to look into the Dump.

Can you please let me know is there any other way we can find out which entity 
is causing this issue or any other tool to analyse and debug the Out of Memory 
Issue to find the exact entity is causing this issue.

I have attched the entity in dataconfig.xml and heap Anayser screen shot.


Thanks
P.Yuvaraj Kumar 

Re: Java heap space exception in 4.2.1

[J Mohamed Zahoor]

Hprof introspection shows that huge Double Array are using up 75% of heap 
space... which belongs to Lucen's FieldCache..

./zahoor


On 17-May-2013, at 12:47 PM, J Mohamed Zahoor  wrote:

> Hi 
> 
> I moved to 4.2.1 from 4.1 recently.. everything was working fine until i 
> added few more stats query..
> Now i am getting this error frequently that solr does not run even for 2 
> minutes continuously.
> All 5GB is getting used instantaneously in few queries...
> 
> 
> SEVERE: null:java.lang.RuntimeException: java.lang.OutOfMemoryError: Java 
> heap space
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:653)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:366)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
>   at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
>   at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:560)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1072)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1006)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
>   at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
>   at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
>   at org.eclipse.jetty.server.Server.handle(Server.java:365)
>   at 
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:485)
>   at 
> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
>   at 
> org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:937)
>   at 
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:998)
>   at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:856)
>   at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
>   at 
> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
>   at 
> org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
>   at java.lang.Thread.run(Thread.java:722)
> 
> 
> 
> one thing i did was when we moved from 4.1 to 4.2.1 i changed only the 
> solr.war and left other jars and config as it is...
> WIll that create a problem...??
> 
> ./zahoor
> 
> 

Re: Java heap space exception in 4.2.1

[Shawn Heisey]

On 5/17/2013 1:17 AM, J Mohamed Zahoor wrote:
> I moved to 4.2.1 from 4.1 recently.. everything was working fine until i 
> added few more stats query..
> Now i am getting this error frequently that solr does not run even for 2 
> minutes continuously.
> All 5GB is getting used instantaneously in few queries...

Someone on IRC ran into memory problems upgrading from 4.0 to 4.2.  It
wasn't OOM errors, they were just using a lot more heap than they were
before and running into constant full garbage collections.

There is another message on this list about someone who upgraded from
3.5 to 4.2 and is having memory troubles.

The person on IRC made most of their fields unstored and reindexed,
which fixed the problem for them.  They only needed a few fields stored.

Because the IRC user was on 4.0, I originally thought it had something
to do with compressed stored fields, but on this thread, they started
with 4.1.  If that was the released 4.1.0 and not a SNAPSHOT version,
then they had compressed stored fields before the upgrade.

The user on IRC was not using termvectors or docvalues, which would be
potential pain points unique to 4.2.

I'm using 4.2.1 with no trouble in my setup, but I do have a heap that's
considerably larger than I need.  There are no apparent memory leaks -
it's been running for over a month with updates once a minute.  I've
finally switched over from the 3.5.0 index to the new one, so for the
last few days, it has been also taking our full query load.

What could have changed between 4.1 and 4.2 to cause dramatically
increased memory usage?

>From my /admin/system:

2013-04-05T15:52:55.751Z

Thanks,
Shawn

Re: Java heap space exception in 4.2.1

[J Mohamed Zahoor]

Memory increase a lot with queries which have facets… 


./Zahoor


On 17-May-2013, at 10:00 PM, Shawn Heisey  wrote:

> On 5/17/2013 1:17 AM, J Mohamed Zahoor wrote:
>> I moved to 4.2.1 from 4.1 recently.. everything was working fine until i 
>> added few more stats query..
>> Now i am getting this error frequently that solr does not run even for 2 
>> minutes continuously.
>> All 5GB is getting used instantaneously in few queries...
> 
> Someone on IRC ran into memory problems upgrading from 4.0 to 4.2.  It
> wasn't OOM errors, they were just using a lot more heap than they were
> before and running into constant full garbage collections.
> 
> There is another message on this list about someone who upgraded from
> 3.5 to 4.2 and is having memory troubles.
> 
> The person on IRC made most of their fields unstored and reindexed,
> which fixed the problem for them.  They only needed a few fields stored.
> 
> Because the IRC user was on 4.0, I originally thought it had something
> to do with compressed stored fields, but on this thread, they started
> with 4.1.  If that was the released 4.1.0 and not a SNAPSHOT version,
> then they had compressed stored fields before the upgrade.
> 
> The user on IRC was not using termvectors or docvalues, which would be
> potential pain points unique to 4.2.
> 
> I'm using 4.2.1 with no trouble in my setup, but I do have a heap that's
> considerably larger than I need.  There are no apparent memory leaks -
> it's been running for over a month with updates once a minute.  I've
> finally switched over from the 3.5.0 index to the new one, so for the
> last few days, it has been also taking our full query load.
> 
> What could have changed between 4.1 and 4.2 to cause dramatically
> increased memory usage?
> 
> From my /admin/system:
> 
> 2013-04-05T15:52:55.751Z
> 
> Thanks,
> Shawn
> 

Re: Java heap space exception in 4.2.1

[J Mohamed Zahoor]

aah… was doing a facet on a double field which was having 6 decimal places…
No surprise that the lucene cache got full…

.z/ahoor

On 17-May-2013, at 11:56 PM, J Mohamed Zahoor  wrote:

> Memory increase a lot with queries which have facets… 
> 
> 
> ./Zahoor
> 
> 
> On 17-May-2013, at 10:00 PM, Shawn Heisey  wrote:
> 
>> On 5/17/2013 1:17 AM, J Mohamed Zahoor wrote:
>>> I moved to 4.2.1 from 4.1 recently.. everything was working fine until i 
>>> added few more stats query..
>>> Now i am getting this error frequently that solr does not run even for 2 
>>> minutes continuously.
>>> All 5GB is getting used instantaneously in few queries...
>> 
>> Someone on IRC ran into memory problems upgrading from 4.0 to 4.2.  It
>> wasn't OOM errors, they were just using a lot more heap than they were
>> before and running into constant full garbage collections.
>> 
>> There is another message on this list about someone who upgraded from
>> 3.5 to 4.2 and is having memory troubles.
>> 
>> The person on IRC made most of their fields unstored and reindexed,
>> which fixed the problem for them.  They only needed a few fields stored.
>> 
>> Because the IRC user was on 4.0, I originally thought it had something
>> to do with compressed stored fields, but on this thread, they started
>> with 4.1.  If that was the released 4.1.0 and not a SNAPSHOT version,
>> then they had compressed stored fields before the upgrade.
>> 
>> The user on IRC was not using termvectors or docvalues, which would be
>> potential pain points unique to 4.2.
>> 
>> I'm using 4.2.1 with no trouble in my setup, but I do have a heap that's
>> considerably larger than I need.  There are no apparent memory leaks -
>> it's been running for over a month with updates once a minute.  I've
>> finally switched over from the 3.5.0 index to the new one, so for the
>> last few days, it has been also taking our full query load.
>> 
>> What could have changed between 4.1 and 4.2 to cause dramatically
>> increased memory usage?
>> 
>> From my /admin/system:
>> 
>> 2013-04-05T15:52:55.751Z
>> 
>> Thanks,
>> Shawn
>> 
> 

Re: Java heap space exception in 4.2.1

[Jam Luo]

I have the same problem. at 4.1  ,a solr instance could take 8,000,000,000
doc. but at 4.2.1, a instance only take 400,000,000 doc, it will oom at
facet query.  the facet field was token by space.

May 27, 2013 11:12:55 AM org.apache.solr.common.SolrException log
SEVERE: null:java.lang.RuntimeException: java.lang.OutOfMemoryError: Java
heap space
at
org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:653)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:366)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1338)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
at org.eclipse.jetty.server.Server.handle(Server.java:350)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)
at
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:900)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:954)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:851)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66)
at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:603)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:538)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.OutOfMemoryError: Java heap space
at
org.apache.lucene.index.DocTermOrds.uninvert(DocTermOrds.java:448)
at
org.apache.solr.request.UnInvertedField.(UnInvertedField.java:179)
at
org.apache.solr.request.UnInvertedField.getUnInvertedField(UnInvertedField.java:664)
at
org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:426)
at
org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:517)
at
org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:252)
at
org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:78)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1825)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:639)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1338)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
at

Re: Java heap space exception in 4.2.1

[Jam Luo]

I am sorry about a type mistake  8,000,000,000 -> 800,000,000


2013/5/27 Jam Luo 

> I have the same problem. at 4.1  ,a solr instance could take 8,000,000,000
> doc. but at 4.2.1, a instance only take 400,000,000 doc, it will oom at
> facet query.  the facet field was token by space.
>
> May 27, 2013 11:12:55 AM org.apache.solr.common.SolrException log
> SEVERE: null:java.lang.RuntimeException: java.lang.OutOfMemoryError: Java
> heap space
> at
> org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:653)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:366)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1338)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
> at org.eclipse.jetty.server.Server.handle(Server.java:350)
> at
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
> at
> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)
> at
> org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:900)
> at
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:954)
> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:851)
> at
> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
> at
> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66)
> at
> org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:603)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:538)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.lucene.index.DocTermOrds.uninvert(DocTermOrds.java:448)
> at
> org.apache.solr.request.UnInvertedField.(UnInvertedField.java:179)
> at
> org.apache.solr.request.UnInvertedField.getUnInvertedField(UnInvertedField.java:664)
> at
> org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:426)
> at
> org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:517)
> at
> org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:252)
> at
> org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:78)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1825)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:639)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1338)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
> at
>

Re: Java heap space exception in 4.2.1

[Erick Erickson]

400M docs is quite a large number of documents for a single piece of
hardware, and
if you're faceting over a large number of unique values, this will
chew up memory.

So it's not surprising that you're seeing OOMs, I suspect you just have too many
documents on a single machine..

Best
Erick


On Mon, May 27, 2013 at 3:11 AM, Jam Luo  wrote:
> I am sorry about a type mistake  8,000,000,000 -> 800,000,000
>
>
> 2013/5/27 Jam Luo 
>
>> I have the same problem. at 4.1  ,a solr instance could take 8,000,000,000
>> doc. but at 4.2.1, a instance only take 400,000,000 doc, it will oom at
>> facet query.  the facet field was token by space.
>>
>> May 27, 2013 11:12:55 AM org.apache.solr.common.SolrException log
>> SEVERE: null:java.lang.RuntimeException: java.lang.OutOfMemoryError: Java
>> heap space
>> at
>> org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:653)
>> at
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:366)
>> at
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
>> at
>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1338)
>> at
>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
>> at
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
>> at
>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
>> at
>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
>> at
>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
>> at
>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
>> at
>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
>> at
>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
>> at
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
>> at
>> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
>> at
>> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
>> at
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
>> at org.eclipse.jetty.server.Server.handle(Server.java:350)
>> at
>> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
>> at
>> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)
>> at
>> org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:900)
>> at
>> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:954)
>> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:851)
>> at
>> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
>> at
>> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66)
>> at
>> org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254)
>> at
>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:603)
>> at
>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:538)
>> at java.lang.Thread.run(Thread.java:662)
>> Caused by: java.lang.OutOfMemoryError: Java heap space
>> at
>> org.apache.lucene.index.DocTermOrds.uninvert(DocTermOrds.java:448)
>> at
>> org.apache.solr.request.UnInvertedField.(UnInvertedField.java:179)
>> at
>> org.apache.solr.request.UnInvertedField.getUnInvertedField(UnInvertedField.java:664)
>> at
>> org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:426)
>> at
>> org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:517)
>> at
>> org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:252)
>> at
>> org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:78)
>> at
>> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208)
>> at
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
>> at org.apach

Delta import throws java heap space exception

[Richard Marquina Lopez]

Hi,

I have some problems when execute the delta import with 2 million of rows
from mysql database:

java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapCharBuffer.(HeapCharBuffer.java:57)
at java.nio.CharBuffer.allocate(CharBuffer.java:331)
at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:777)
at java.nio.charset.Charset.decode(Charset.java:810)
at com.mysql.jdbc.StringUtils.toString(StringUtils.java:2010)
at com.mysql.jdbc.ResultSetRow.getString(ResultSetRow.java:820)
at com.mysql.jdbc.BufferRow.getString(BufferRow.java:541)
at com.mysql.jdbc.ResultSetImpl.getStringInternal(
ResultSetImpl.java:5812)
at com.mysql.jdbc.ResultSetImpl.getString(ResultSetImpl.java:5689)
at com.mysql.jdbc.ResultSetImpl.getObject(ResultSetImpl.java:4986)
at com.mysql.jdbc.ResultSetImpl.getObject(ResultSetImpl.java:5175)
at org.apache.solr.handler.dataimport.JdbcDataSource$
ResultSetIterator.getARow(JdbcDataSource.java:315)
at org.apache.solr.handler.dataimport.JdbcDataSource$
ResultSetIterator.access$700(JdbcDataSource.java:254)
at org.apache.solr.handler.dataimport.JdbcDataSource$
ResultSetIterator$1.next(JdbcDataSource.java:294)
at org.apache.solr.handler.dataimport.JdbcDataSource$
ResultSetIterator$1.next(JdbcDataSource.java:286)
at org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(
EntityProcessorBase.java:117)
at org.apache.solr.handler.dataimport.SqlEntityProcessor.
nextModifiedRowKey(SqlEntityProcessor.java:86)
at org.apache.solr.handler.dataimport.EntityProcessorWrapper.
nextModifiedRowKey(EntityProcessorWrapper.java:267)
at org.apache.solr.handler.dataimport.DocBuilder.
collectDelta(DocBuilder.java:781)
at org.apache.solr.handler.dataimport.DocBuilder.doDelta(
DocBuilder.java:338)
at org.apache.solr.handler.dataimport.DocBuilder.execute(
DocBuilder.java:223)
at org.apache.solr.handler.dataimport.DataImporter.
doDeltaImport(DataImporter.java:440)
at org.apache.solr.handler.dataimport.DataImporter.
runCmd(DataImporter.java:478)
at org.apache.solr.handler.dataimport.DataImporter$1.run(
DataImporter.java:457)

--

java.sql.SQLException: Streaming result set
com.mysql.jdbc.RowDataDynamic@47a034e7
is still active.
No statements may be issued when any streaming result sets are open and in
use on a given connection.
Ensure that you have called .close() on any active streaming result sets
before attempting more queries.
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:927)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:924)
at com.mysql.jdbc.MysqlIO.checkForOutstandingStreamingDa
ta(MysqlIO.java:3361)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2524)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2778)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2828)
at com.mysql.jdbc.ConnectionImpl.rollbackNoChecks(
ConnectionImpl.java:5204)
at com.mysql.jdbc.ConnectionImpl.rollback(ConnectionImpl.java:5087)
at com.mysql.jdbc.ConnectionImpl.realClose(ConnectionImpl.java:4690)
at com.mysql.jdbc.ConnectionImpl.close(ConnectionImpl.java:1649)
at org.apache.solr.handler.dataimport.JdbcDataSource.
closeConnection(JdbcDataSource.java:436)
at org.apache.solr.handler.dataimport.JdbcDataSource.
close(JdbcDataSource.java:421)
at org.apache.solr.handler.dataimport.DocBuilder.
closeEntityProcessorWrappers(DocBuilder.java:288)
at org.apache.solr.handler.dataimport.DocBuilder.execute(
DocBuilder.java:277)
at org.apache.solr.handler.dataimport.DataImporter.
doDeltaImport(DataImporter.java:440)
at org.apache.solr.handler.dataimport.DataImporter.
runCmd(DataImporter.java:478)
at org.apache.solr.handler.dataimport.DataImporter$1.run(
DataImporter.java:457)

Currently I have the batchSize parameter stetted to -1

Configuration:
- SOLR 4.4
- Centos 5.5
- 2GB RAM
- 1 Procesosr

Does someone have the same error?
Could someone help me, please?

Thank you,
Richard

Re: java heap space error when faceting

[Yonik Seeley]

On Sat, Jan 16, 2010 at 10:01 AM, Matt Mitchell  wrote:
> I have an index with more than 6 million docs. All is well, until I turn on
> faceting and specify a facet.field. There is only about unique 20 values for
> this particular facet throughout the entire index.

Hmmm, that doesn't sound right... unless you're already near max
memory usage due to other things.
Is this a single-valued or multi-valued field?  If multi-valued, how
many values does each document have on average?

-Yonik
http://www.lucidimagination.com

Re: java heap space error when faceting

[Matt Mitchell]

These are single valued fields. Strings and integers. Is there more specific
info I could post to help diagnose what might be happening?
Thanks!
Matt

On Sat, Jan 16, 2010 at 10:42 AM, Yonik Seeley
wrote:

> On Sat, Jan 16, 2010 at 10:01 AM, Matt Mitchell 
> wrote:
> > I have an index with more than 6 million docs. All is well, until I turn
> on
> > faceting and specify a facet.field. There is only about unique 20 values
> for
> > this particular facet throughout the entire index.
>
> Hmmm, that doesn't sound right... unless you're already near max
> memory usage due to other things.
> Is this a single-valued or multi-valued field?  If multi-valued, how
> many values does each document have on average?
>
> -Yonik
> http://www.lucidimagination.com
>