Sure I will try to do that.. and will get back to you Sent from my iPhone
> On 16-Aug-2019, at 6:23 PM, Michael Cizmar <[email protected]> wrote: > > Priya - Was this right? > > 48GB and 1-Core Intel(R) Xeon(R) CPU > > While not directly related to an out of memory issue, you should have more > cores allocated to ES. > >> On Fri, Aug 16, 2019 at 1:09 AM Priya Arora <[email protected]> wrote: >> >> *Existing Threads/connections configuration is :-* >> >> How many worker threads do you have? - 15 worker threads has been >> allocated(in properties.xml file). >> And the Tika Extractor connections -10 connections are defined. >> >> Is this suggested to reduce the number more. >> If not, what else can be a solution >> >> Thanks >> Priya >> >> >> >>> On Wed, Aug 14, 2019 at 5:32 PM Karl Wright <[email protected]> wrote: >>> >>> How many worker threads do you have? >>> Even if each worker thread is constrained in memory, and they should be, >>> you can easily cause things to run out of memory by giving too many >> worker >>> threads. Another way to keep Tika's usage constrained would be to reduce >>> the number of Tika Extractor connections, because that effectively limits >>> the number of extractions that can be going on at the same time. >>> >>> Karl >>> >>> >>>> On Wed, Aug 14, 2019 at 7:23 AM Priya Arora <[email protected]> wrote: >>>> >>>> Yes , I am using Tika Extractor. And the version used for manifold is >>> 2.13. >>>> Also I am using postgres as database. >>>> >>>> I have 4 types of jobs >>>> One is accessing/re crawling data from a public site. Other three are >>>> accessing intranet site. >>>> Out of which two are giving me correct output-without any error and >> third >>>> one which is having data more than the other two , and giving me this >>>> error. >>>> >>>> Is there any possibility with site accessibility issue. Can you please >>>> suggest some solution >>>> Thanks and regards >>>> Priya >>>> >>>> On Wed, Aug 14, 2019 at 3:11 PM Karl Wright <[email protected]> >> wrote: >>>> >>>>> I will need to know more. Do you have the tika extractor in your >>>>> pipeline? If so, what version of ManifoldCF are you using? Tika has >>> had >>>>> bugs related to memory consumption in the past; the out of memory >>>> exception >>>>> may be coming from it and therefore a stack trace is critical to >> have. >>>>> >>>>> Alternatively, you can upgrade to the latest version of MCF (2.13) >> and >>>> that >>>>> has a newer version of Tika without those problem. But you may need >> to >>>> get >>>>> the agents process more memory. >>>>> >>>>> Another possible cause is that you're using hsqldb in production. >>> HSQLDB >>>>> keeps all of its tables in memory. If you have a large crawl, you do >>> not >>>>> want to use HSQLDB. >>>>> >>>>> Thanks, >>>>> Karl >>>>> >>>>> >>>>> On Wed, Aug 14, 2019 at 3:41 AM Priya Arora <[email protected]> >>> wrote: >>>>> >>>>>> Hi Karl, >>>>>> >>>>>> Manifold CF logs hints out me an error like : >>>>>> agents process ran out of memory - shutting down >>>>>> java.lang.OutOfMemoryError: Java heap space >>>>>> >>>>>> Also I have -Xms1024m ,-Xmx1024m memory allocated in >>>>>> start-options.env.unix, start-options.env.win file. >>>>>> Also Configuration:- >>>>>> 1) For Crawler server - 16 GB RAM and 8-Core Intel(R) Xeon(R) CPU >>>> E5-2660 >>>>>> v3 @ 2.60GHz and >>>>>> >>>>>> 2) For Elasticsearch server - 48GB and 1-Core Intel(R) Xeon(R) CPU >>>>> E5-2660 >>>>>> v3 @ 2.60GHz and i am using postgres as database. >>>>>> >>>>>> Can you please help me out, what to do in this case. >>>>>> >>>>>> Thanks >>>>>> Priya >>>>>> >>>>>> >>>>>> On Wed, Aug 14, 2019 at 12:33 PM Karl Wright <[email protected]> >>>> wrote: >>>>>> >>>>>>> The error occurs, I believe, as the result of basic connection >>>>> problems, >>>>>>> e.g. the connection is getting rejected. You can find more >>>> information >>>>>> in >>>>>>> the simple history, and in the manifoldcf log. >>>>>>> >>>>>>> I would like to know the underlying cause, since the connector >>> should >>>>> be >>>>>>> resilient against errors of this kind. >>>>>>> >>>>>>> Karl >>>>>>> >>>>>>> >>>>>>> On Wed, Aug 14, 2019, 1:46 AM Priya Arora <[email protected]> >>>> wrote: >>>>>>> >>>>>>>> Hi Karl, >>>>>>>> >>>>>>>> I have an web Repository connector(Seeds:- an intranet Site)., >>> and >>>>> job >>>>>> i >>>>>>>> son Production server. >>>>>>>> >>>>>>>> When i ran job on PROD, the job stops itself 2 times with and >>>>>>> error:Error: >>>>>>>> Unexpected HTTP result code: -1: null. >>>>>>>> >>>>>>>> >>>>>>>> Can you please provide me an idea, in which it happens so? >>>>>>>> >>>>>>>> Thanks and regards >>>>>>>> Priya Arora >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >>
