[jira] [Commented] (CONNECTORS-1105) Add maven delivery targets to poms
[ https://issues.apache.org/jira/browse/CONNECTORS-1105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907559#comment-16907559 ] Karl Wright commented on CONNECTORS-1105: - Feel free to add the linting fix to the poms that need it. It's probably a lot faster than fixing the javadoc itself for the moment. > Add maven delivery targets to poms > -- > > Key: CONNECTORS-1105 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1105 > Project: ManifoldCF > Issue Type: Improvement > Components: Build >Affects Versions: ManifoldCF 1.8 >Reporter: Karl Wright >Assignee: Markus Schuch >Priority: Major > Fix For: ManifoldCF next > > > We've been asked to deliver mcf jars and wars to maven central repository by > some developers. This ticket represents that work. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (CONNECTORS-1105) Add maven delivery targets to poms
[
https://issues.apache.org/jira/browse/CONNECTORS-1105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907557#comment-16907557
]
Markus Schuch commented on CONNECTORS-1105:
---
The javadoc generation fails due to linting errors.
Linting can be disabled with
{code}
org.apache.maven.plugins
maven-javadoc-plugin
-Xdoclint:none
{code}
> Add maven delivery targets to poms
> --
>
> Key: CONNECTORS-1105
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1105
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Build
>Affects Versions: ManifoldCF 1.8
>Reporter: Karl Wright
>Assignee: Markus Schuch
>Priority: Major
> Fix For: ManifoldCF next
>
>
> We've been asked to deliver mcf jars and wars to maven central repository by
> some developers. This ticket represents that work.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (CONNECTORS-1105) Add maven delivery targets to poms
[ https://issues.apache.org/jira/browse/CONNECTORS-1105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907540#comment-16907540 ] Markus Schuch commented on CONNECTORS-1105: --- Nexus setup is ready. I am able to upload snapshots now: https://repository.apache.org/content/repositories/snapshots/org/apache/manifoldcf/ (upload still running...) > Add maven delivery targets to poms > -- > > Key: CONNECTORS-1105 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1105 > Project: ManifoldCF > Issue Type: Improvement > Components: Build >Affects Versions: ManifoldCF 1.8 >Reporter: Karl Wright >Assignee: Markus Schuch >Priority: Major > Fix For: ManifoldCF next > > > We've been asked to deliver mcf jars and wars to maven central repository by > some developers. This ticket represents that work. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
Re: Unexpected HTTP result code: -1: null
How many worker threads do you have? Even if each worker thread is constrained in memory, and they should be, you can easily cause things to run out of memory by giving too many worker threads. Another way to keep Tika's usage constrained would be to reduce the number of Tika Extractor connections, because that effectively limits the number of extractions that can be going on at the same time. Karl On Wed, Aug 14, 2019 at 7:23 AM Priya Arora wrote: > Yes , I am using Tika Extractor. And the version used for manifold is 2.13. > Also I am using postgres as database. > > I have 4 types of jobs > One is accessing/re crawling data from a public site. Other three are > accessing intranet site. > Out of which two are giving me correct output-without any error and third > one which is having data more than the other two , and giving me this > error. > > Is there any possibility with site accessibility issue. Can you please > suggest some solution > Thanks and regards > Priya > > On Wed, Aug 14, 2019 at 3:11 PM Karl Wright wrote: > > > I will need to know more. Do you have the tika extractor in your > > pipeline? If so, what version of ManifoldCF are you using? Tika has had > > bugs related to memory consumption in the past; the out of memory > exception > > may be coming from it and therefore a stack trace is critical to have. > > > > Alternatively, you can upgrade to the latest version of MCF (2.13) and > that > > has a newer version of Tika without those problem. But you may need to > get > > the agents process more memory. > > > > Another possible cause is that you're using hsqldb in production. HSQLDB > > keeps all of its tables in memory. If you have a large crawl, you do not > > want to use HSQLDB. > > > > Thanks, > > Karl > > > > > > On Wed, Aug 14, 2019 at 3:41 AM Priya Arora wrote: > > > > > Hi Karl, > > > > > > Manifold CF logs hints out me an error like : > > > agents process ran out of memory - shutting down > > > java.lang.OutOfMemoryError: Java heap space > > > > > > Also I have -Xms1024m ,-Xmx1024m memory allocated in > > > start-options.env.unix, start-options.env.win file. > > > Also Configuration:- > > > 1) For Crawler server - 16 GB RAM and 8-Core Intel(R) Xeon(R) CPU > E5-2660 > > > v3 @ 2.60GHz and > > > > > > 2) For Elasticsearch server - 48GB and 1-Core Intel(R) Xeon(R) CPU > > E5-2660 > > > v3 @ 2.60GHz and i am using postgres as database. > > > > > > Can you please help me out, what to do in this case. > > > > > > Thanks > > > Priya > > > > > > > > > On Wed, Aug 14, 2019 at 12:33 PM Karl Wright > wrote: > > > > > > > The error occurs, I believe, as the result of basic connection > > problems, > > > > e.g. the connection is getting rejected. You can find more > information > > > in > > > > the simple history, and in the manifoldcf log. > > > > > > > > I would like to know the underlying cause, since the connector should > > be > > > > resilient against errors of this kind. > > > > > > > > Karl > > > > > > > > > > > > On Wed, Aug 14, 2019, 1:46 AM Priya Arora > wrote: > > > > > > > > > Hi Karl, > > > > > > > > > > I have an web Repository connector(Seeds:- an intranet Site)., and > > job > > > i > > > > > son Production server. > > > > > > > > > > When i ran job on PROD, the job stops itself 2 times with and > > > > error:Error: > > > > > Unexpected HTTP result code: -1: null. > > > > > > > > > > > > > > > Can you please provide me an idea, in which it happens so? > > > > > > > > > > Thanks and regards > > > > > Priya Arora > > > > > > > > > > > > > > >
Re: Unexpected HTTP result code: -1: null
Yes , I am using Tika Extractor. And the version used for manifold is 2.13. Also I am using postgres as database. I have 4 types of jobs One is accessing/re crawling data from a public site. Other three are accessing intranet site. Out of which two are giving me correct output-without any error and third one which is having data more than the other two , and giving me this error. Is there any possibility with site accessibility issue. Can you please suggest some solution Thanks and regards Priya On Wed, Aug 14, 2019 at 3:11 PM Karl Wright wrote: > I will need to know more. Do you have the tika extractor in your > pipeline? If so, what version of ManifoldCF are you using? Tika has had > bugs related to memory consumption in the past; the out of memory exception > may be coming from it and therefore a stack trace is critical to have. > > Alternatively, you can upgrade to the latest version of MCF (2.13) and that > has a newer version of Tika without those problem. But you may need to get > the agents process more memory. > > Another possible cause is that you're using hsqldb in production. HSQLDB > keeps all of its tables in memory. If you have a large crawl, you do not > want to use HSQLDB. > > Thanks, > Karl > > > On Wed, Aug 14, 2019 at 3:41 AM Priya Arora wrote: > > > Hi Karl, > > > > Manifold CF logs hints out me an error like : > > agents process ran out of memory - shutting down > > java.lang.OutOfMemoryError: Java heap space > > > > Also I have -Xms1024m ,-Xmx1024m memory allocated in > > start-options.env.unix, start-options.env.win file. > > Also Configuration:- > > 1) For Crawler server - 16 GB RAM and 8-Core Intel(R) Xeon(R) CPU E5-2660 > > v3 @ 2.60GHz and > > > > 2) For Elasticsearch server - 48GB and 1-Core Intel(R) Xeon(R) CPU > E5-2660 > > v3 @ 2.60GHz and i am using postgres as database. > > > > Can you please help me out, what to do in this case. > > > > Thanks > > Priya > > > > > > On Wed, Aug 14, 2019 at 12:33 PM Karl Wright wrote: > > > > > The error occurs, I believe, as the result of basic connection > problems, > > > e.g. the connection is getting rejected. You can find more information > > in > > > the simple history, and in the manifoldcf log. > > > > > > I would like to know the underlying cause, since the connector should > be > > > resilient against errors of this kind. > > > > > > Karl > > > > > > > > > On Wed, Aug 14, 2019, 1:46 AM Priya Arora wrote: > > > > > > > Hi Karl, > > > > > > > > I have an web Repository connector(Seeds:- an intranet Site)., and > job > > i > > > > son Production server. > > > > > > > > When i ran job on PROD, the job stops itself 2 times with and > > > error:Error: > > > > Unexpected HTTP result code: -1: null. > > > > > > > > > > > > Can you please provide me an idea, in which it happens so? > > > > > > > > Thanks and regards > > > > Priya Arora > > > > > > > > > >
Re: Unexpected HTTP result code: -1: null
I will need to know more. Do you have the tika extractor in your pipeline? If so, what version of ManifoldCF are you using? Tika has had bugs related to memory consumption in the past; the out of memory exception may be coming from it and therefore a stack trace is critical to have. Alternatively, you can upgrade to the latest version of MCF (2.13) and that has a newer version of Tika without those problem. But you may need to get the agents process more memory. Another possible cause is that you're using hsqldb in production. HSQLDB keeps all of its tables in memory. If you have a large crawl, you do not want to use HSQLDB. Thanks, Karl On Wed, Aug 14, 2019 at 3:41 AM Priya Arora wrote: > Hi Karl, > > Manifold CF logs hints out me an error like : > agents process ran out of memory - shutting down > java.lang.OutOfMemoryError: Java heap space > > Also I have -Xms1024m ,-Xmx1024m memory allocated in > start-options.env.unix, start-options.env.win file. > Also Configuration:- > 1) For Crawler server - 16 GB RAM and 8-Core Intel(R) Xeon(R) CPU E5-2660 > v3 @ 2.60GHz and > > 2) For Elasticsearch server - 48GB and 1-Core Intel(R) Xeon(R) CPU E5-2660 > v3 @ 2.60GHz and i am using postgres as database. > > Can you please help me out, what to do in this case. > > Thanks > Priya > > > On Wed, Aug 14, 2019 at 12:33 PM Karl Wright wrote: > > > The error occurs, I believe, as the result of basic connection problems, > > e.g. the connection is getting rejected. You can find more information > in > > the simple history, and in the manifoldcf log. > > > > I would like to know the underlying cause, since the connector should be > > resilient against errors of this kind. > > > > Karl > > > > > > On Wed, Aug 14, 2019, 1:46 AM Priya Arora wrote: > > > > > Hi Karl, > > > > > > I have an web Repository connector(Seeds:- an intranet Site)., and job > i > > > son Production server. > > > > > > When i ran job on PROD, the job stops itself 2 times with and > > error:Error: > > > Unexpected HTTP result code: -1: null. > > > > > > > > > Can you please provide me an idea, in which it happens so? > > > > > > Thanks and regards > > > Priya Arora > > > > > >
Re: Unexpected HTTP result code: -1: null
Hi Karl, Manifold CF logs hints out me an error like : agents process ran out of memory - shutting down java.lang.OutOfMemoryError: Java heap space Also I have -Xms1024m ,-Xmx1024m memory allocated in start-options.env.unix, start-options.env.win file. Also Configuration:- 1) For Crawler server - 16 GB RAM and 8-Core Intel(R) Xeon(R) CPU E5-2660 v3 @ 2.60GHz and 2) For Elasticsearch server - 48GB and 1-Core Intel(R) Xeon(R) CPU E5-2660 v3 @ 2.60GHz and i am using postgres as database. Can you please help me out, what to do in this case. Thanks Priya On Wed, Aug 14, 2019 at 12:33 PM Karl Wright wrote: > The error occurs, I believe, as the result of basic connection problems, > e.g. the connection is getting rejected. You can find more information in > the simple history, and in the manifoldcf log. > > I would like to know the underlying cause, since the connector should be > resilient against errors of this kind. > > Karl > > > On Wed, Aug 14, 2019, 1:46 AM Priya Arora wrote: > > > Hi Karl, > > > > I have an web Repository connector(Seeds:- an intranet Site)., and job i > > son Production server. > > > > When i ran job on PROD, the job stops itself 2 times with and > error:Error: > > Unexpected HTTP result code: -1: null. > > > > > > Can you please provide me an idea, in which it happens so? > > > > Thanks and regards > > Priya Arora > > >
Re: Unexpected HTTP result code: -1: null
The error occurs, I believe, as the result of basic connection problems, e.g. the connection is getting rejected. You can find more information in the simple history, and in the manifoldcf log. I would like to know the underlying cause, since the connector should be resilient against errors of this kind. Karl On Wed, Aug 14, 2019, 1:46 AM Priya Arora wrote: > Hi Karl, > > I have an web Repository connector(Seeds:- an intranet Site)., and job i > son Production server. > > When i ran job on PROD, the job stops itself 2 times with and error:Error: > Unexpected HTTP result code: -1: null. > > > Can you please provide me an idea, in which it happens so? > > Thanks and regards > Priya Arora >
