Re: Unexpected HTTP result code: -1: null

2019-08-14 Thread Karl Wright
The error occurs, I believe, as the result of basic connection problems, e.g. the connection is getting rejected. You can find more information in the simple history, and in the manifoldcf log. I would like to know the underlying cause, since the connector should be resilient against errors of

[jira] [Commented] (CONNECTORS-1105) Add maven delivery targets to poms

2019-08-13 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906746#comment-16906746 ] Karl Wright commented on CONNECTORS-1105: - All that I know is summarized in this ticket

[jira] [Resolved] (CONNECTORS-1591) RTF comment parsing problem

2019-08-13 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright resolved CONNECTORS-1591. - Resolution: Fixed r1865081 > RTF comment parsing prob

[jira] [Commented] (CONNECTORS-1591) RTF comment parsing problem

2019-08-13 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906672#comment-16906672 ] Karl Wright commented on CONNECTORS-1591: - Ok, I'll look into this update shortly. >

Re: Reminder: August 31st is the next scheduled ManifoldCF release

2019-08-13 Thread Karl Wright
> The other tickets marked for 2.14 seem to wait on external resources. > > In addition i just resumed > https://issues.apache.org/jira/browse/CONNECTORS-1105. May be we get > this done and can add this to the release. > > Markus > > Am 12.08.2019 um 14:57 schrieb Karl Wr

Reminder: August 31st is the next scheduled ManifoldCF release

2019-08-12 Thread Karl Wright
I had hoped that we could finish the OpenText Content Service/Web Service connector by this release cycle but I do not think it will be finished. So I suggest we go ahead with release plans. It's a pretty light release I'm afraid. Thoughts? Karl

Re: Elastic Output Connector SSLException

2019-08-09 Thread Karl Wright
"Connection Reset" sounds like something in the server's SSL configuration is dropping the connection because it doesn't like the protocol that was negotiated. This might be a heavy-handed way of addressing security issues that arose with some ciphers used in SSL a year or two ago, not sure.

[jira] [Resolved] (CONNECTORS-1611) Update MySQL Version

2019-08-06 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright resolved CONNECTORS-1611. - Resolution: Fixed Fix Version/s: ManifoldCF 2.14 r1864515 > Update My

Re: Solr Repository Connector

2019-08-05 Thread Karl Wright
search-index > > For such a scenario, do you think MCF is not the ideal option as the > ETL/ingestion tool? Should I go for a lower-level ETL tool such as Apache > Nifi ? > Or will writing a MCF Solr repository connector be useful to achieve this? > WDYT? > > Thanks a lot. > Regards

Re: Solr Repository Connector

2019-08-05 Thread Karl Wright
If you are trying to extract data from a Solr index, I know of no way to do that. Karl On Mon, Aug 5, 2019 at 9:08 AM Dileepa Jayakody wrote: > Hi All, > > Thanks for your replies. > I'm looking for a repository connector. I've used the Solr output > connector before. But now what I need is to

Re: Solr Repository Connector

2019-08-05 Thread Karl Wright
If you use Solr Cloud, ManifoldCF's Solr Connector should work for you. Karl On Mon, Aug 5, 2019 at 6:18 AM Dileepa Jayakody wrote: > Hi All, > > I'm working on a project which needs to implement a federated search > solution with heterogeneous data repositories. One repository is a Solr >

[jira] [Assigned] (CONNECTORS-1616) Confluence Authority does not handle Confluence API errors

2019-08-01 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright reassigned CONNECTORS-1616: --- Resolution: Fixed Assignee: Karl Wright Fix Version/s

[jira] [Commented] (CONNECTORS-1616) Confluence Authority does not handle Confluence API errors

2019-08-01 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16897946#comment-16897946 ] Karl Wright commented on CONNECTORS-1616: - So, the issue here is that you are giving

[jira] [Commented] (CONNECTORS-1616) Confluence Authority does not handle Confluence API errors

2019-08-01 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16897940#comment-16897940 ] Karl Wright commented on CONNECTORS-1616: - Ok, there are some problems with it. Why did you

[jira] [Commented] (CONNECTORS-1615) Bad Error Message when IDCOLUMN's value is actually null

2019-07-31 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16897372#comment-16897372 ] Karl Wright commented on CONNECTORS-1615: - Patches welcome. > Bad Error Message w

[jira] [Commented] (CONNECTORS-1615) Bad Error Message when IDCOLUMN's value is actually null

2019-07-31 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16897343#comment-16897343 ] Karl Wright commented on CONNECTORS-1615: - Right, but as I said, I have no way of detecting

[jira] [Commented] (CONNECTORS-1616) Confluence Authority does not handle Confluence API errors

2019-07-31 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16897105#comment-16897105 ] Karl Wright commented on CONNECTORS-1616: - Patches welcome. I'm not the author

[jira] [Assigned] (CONNECTORS-1616) Confluence Authority does not handle Confluence API errors

2019-07-31 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright reassigned CONNECTORS-1616: --- Assignee: Karl Wright > Confluence Authority does not handle Confluence

[jira] [Resolved] (CONNECTORS-1615) Bad Error Message when IDCOLUMN's value is actually null

2019-07-31 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright resolved CONNECTORS-1615. - Resolution: Won't Fix Fix Version/s: ManifoldCF 2.14 > Bad Error Mess

[jira] [Assigned] (CONNECTORS-1615) Bad Error Message when IDCOLUMN's value is actually null

2019-07-31 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright reassigned CONNECTORS-1615: --- Assignee: Karl Wright > Bad Error Message when IDCOLUMN's value is actua

[jira] [Commented] (CONNECTORS-1615) Bad Error Message when IDCOLUMN's value is actually null

2019-07-31 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16897066#comment-16897066 ] Karl Wright commented on CONNECTORS-1615: - Your query should never return rows that have

Re: Solr Output Connector - Too big metadata names

2019-07-25 Thread Karl Wright
em ourselves in the Solr output > connector. What do you think ? > > Julien > > -Message d'origine- > De : Karl Wright > Envoyé : mercredi 19 juin 2019 22:45 > À : dev > Objet : Re: Solr Output Connector - Too big metadata names > > Hi Julien, > > The

[jira] [Commented] (CONNECTORS-1566) Develop CSWS connector as a replacement for deprecated LiveLink LAPI connector

2019-07-19 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16888581#comment-16888581 ] Karl Wright commented on CONNECTORS-1566: - The TLSClientParameters programmatic way

[jira] [Commented] (CONNECTORS-1566) Develop CSWS connector as a replacement for deprecated LiveLink LAPI connector

2019-07-19 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16888567#comment-16888567 ] Karl Wright commented on CONNECTORS-1566: - It sounds like the standard solution is to enable

[jira] [Comment Edited] (CONNECTORS-1566) Develop CSWS connector as a replacement for deprecated LiveLink LAPI connector

2019-07-19 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16888557#comment-16888557 ] Karl Wright edited comment on CONNECTORS-1566 at 7/19/19 6:22 AM

[jira] [Commented] (CONNECTORS-1566) Develop CSWS connector as a replacement for deprecated LiveLink LAPI connector

2019-07-19 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16888561#comment-16888561 ] Karl Wright commented on CONNECTORS-1566: - Here's the CXF documentation on the async

[jira] [Commented] (CONNECTORS-1566) Develop CSWS connector as a replacement for deprecated LiveLink LAPI connector

2019-07-19 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16888557#comment-16888557 ] Karl Wright commented on CONNECTORS-1566: - So, here's where things stand. (1) The checked

Re: Reg. Manifold Indexing performance

2019-07-17 Thread Karl Wright
Hi Praveen, If there is a broken query plan, it will show up in the ManifoldCF log; any query that takes more than 60 seconds to run gets dumped and explained. So it should be possible to rule that out with low effort. The kind of situation I have seen with very large document jobs is that

[jira] [Resolved] (CONNECTORS-1614) UI bug on parameters deletion on Generic Connector

2019-07-17 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright resolved CONNECTORS-1614. - Resolution: Fixed Fix Version/s: ManifoldCF 2.14 r1863226 > UI

[jira] [Assigned] (CONNECTORS-1614) UI bug on parameters deletion on Generic Connector

2019-07-17 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright reassigned CONNECTORS-1614: --- Assignee: Karl Wright > UI bug on parameters deletion on Generic Connec

[jira] [Commented] (CONNECTORS-1566) Develop CSWS connector as a replacement for deprecated LiveLink LAPI connector

2019-07-17 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886919#comment-16886919 ] Karl Wright commented on CONNECTORS-1566: - I had a look at the class com/sun/xml/ws/wsdl

[jira] [Comment Edited] (CONNECTORS-1566) Develop CSWS connector as a replacement for deprecated LiveLink LAPI connector

2019-07-17 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886745#comment-16886745 ] Karl Wright edited comment on CONNECTORS-1566 at 7/17/19 7:32 AM

[jira] [Commented] (CONNECTORS-1566) Develop CSWS connector as a replacement for deprecated LiveLink LAPI connector

2019-07-17 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886745#comment-16886745 ] Karl Wright commented on CONNECTORS-1566: - Ok, updated in svn. Trying locally now

[jira] [Commented] (CONNECTORS-1566) Develop CSWS connector as a replacement for deprecated LiveLink LAPI connector

2019-07-17 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886722#comment-16886722 ] Karl Wright commented on CONNECTORS-1566: - [~kishorekumar], can you verify the current

[jira] [Comment Edited] (CONNECTORS-1566) Develop CSWS connector as a replacement for deprecated LiveLink LAPI connector

2019-07-17 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886710#comment-16886710 ] Karl Wright edited comment on CONNECTORS-1566 at 7/17/19 6:24 AM

[jira] [Commented] (CONNECTORS-1566) Develop CSWS connector as a replacement for deprecated LiveLink LAPI connector

2019-07-17 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886710#comment-16886710 ] Karl Wright commented on CONNECTORS-1566: - I'm thinking that the resource loader being

[jira] [Commented] (CONNECTORS-1566) Develop CSWS connector as a replacement for deprecated LiveLink LAPI connector

2019-07-16 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886576#comment-16886576 ] Karl Wright commented on CONNECTORS-1566: - I am still getting the same error. I think

[jira] [Commented] (CONNECTORS-1566) Develop CSWS connector as a replacement for deprecated LiveLink LAPI connector

2019-07-16 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886541#comment-16886541 ] Karl Wright commented on CONNECTORS-1566: - thanks! implemented. Will debug to the next

[jira] [Commented] (CONNECTORS-1566) Develop CSWS connector as a replacement for deprecated LiveLink LAPI connector

2019-07-16 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886456#comment-16886456 ] Karl Wright commented on CONNECTORS-1566: - I tried implementing this: https

Re: Reg. unstable Manifold instance

2019-07-16 Thread Karl Wright
wrote: > >> Are there some errors or anything interest in the log? >> >> >> >> -- >> >> Michael Cizmar >> >> >> >> >> >> *From: *Karl Wright >> *Reply-To: *"user@manifoldcf.apache.org" >> *Date: *Mond

[jira] [Commented] (CONNECTORS-1566) Develop CSWS connector as a replacement for deprecated LiveLink LAPI connector

2019-07-16 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886308#comment-16886308 ] Karl Wright commented on CONNECTORS-1566: - Currently, the UI fails in the following way

[jira] [Commented] (CONNECTORS-1566) Develop CSWS connector as a replacement for deprecated LiveLink LAPI connector

2019-07-16 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886309#comment-16886309 ] Karl Wright commented on CONNECTORS-1566: - [~rafaharo], do you know offhand a solution

Re: Documentum connection not working

2019-07-16 Thread Karl Wright
Are you running the documentum connector sidecar processes? You need to be running those, and the documentum_server process must include a valid DFC distribution with a valid configuration file. This is where the documentum server name comes from. The documentation for "how to build and deploy"

[jira] [Commented] (CONNECTORS-1566) Develop CSWS connector as a replacement for deprecated LiveLink LAPI connector

2019-07-15 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885287#comment-16885287 ] Karl Wright commented on CONNECTORS-1566: - Hi [~schuch], CONNECTORS-1117 is indeed

Re: Reg. unstable Manifold instance

2019-07-15 Thread Karl Wright
I have heard of this issue before. The app server is what is giving back the 404 errors. I wonder if the version of jetty we ship has a resource leak of some kind. Karl On Sun, Jul 14, 2019 at 11:34 PM Praveen Bejji wrote: > Hi, > > We have been running manifoldcf for almost 6 months now.

[jira] [Commented] (CONNECTORS-1566) Develop CSWS connector as a replacement for deprecated LiveLink LAPI connector

2019-07-14 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16884718#comment-16884718 ] Karl Wright commented on CONNECTORS-1566: - [~schuch], the connector is almost completed

Re: Some jobs is waiting as "stopping" status

2019-07-13 Thread Karl Wright
, Jul 13, 2019 at 4:39 PM Karl Wright wrote: > >>>>>> > ERROR 2019-07-13T16:20:34,259 (Seeding thread) - Exception tossed: > Unexpected job status: 33 > org.apache.manifoldcf.core.interfaces.ManifoldCFException: Unexpected job > status: 33 > <<<<&

Re: Some jobs is waiting as "stopping" status

2019-07-13 Thread Karl Wright
t; If you are talking about is > https://issues.apache.org/jira/browse/CONNECTORS-1613, my setup doesn't > include this change because I use mfc 2.12. Are you suggesting I use a > trunk version? > > Cihad Güzel > > Karl Wright , 13 Tem 2019 Cmt, 17:27 tarihinde şunu > yazdı:

Re: Some jobs is waiting as "stopping" status

2019-07-13 Thread Karl Wright
gt; >> I'm waiting for over an hour for the jdbc job to stop. I have not any >> error logs in my manifolcf log. >> >> Cihad Güzel >> >> Cihad Güzel >> >> >> Karl Wright , 8 Tem 2019 Pzt, 13:23 tarihinde şunu >> yazdı: >> >>&

Re: Some jobs is waiting as "stopping" status

2019-07-08 Thread Karl Wright
pt - start the processes again Thanks, Karl On Mon, Jul 8, 2019 at 5:05 AM Cihad Guzel wrote: > Hi Karl, > > Nothing. I don't have any error log. > > 8 Tem 2019 Pzt 03:18 tarihinde Karl Wright şunu > yazdı: > >> Hi Cihad, >> >> What does your m

Re: Some jobs is waiting as "stopping" status

2019-07-07 Thread Karl Wright
Hi Cihad, What does your manifoldcf log have in it? Any errors? Karl On Sun, Jul 7, 2019 at 3:52 PM Cihad Guzel wrote: > Hi Karl, > > I mistakenly wrote "Stopping" instead of "Aborting". My job is waiting as > "Aborting" status. I have also the same problem while restarting. I am > waiting

Re: manifoldCF and sitemap

2019-07-04 Thread Karl Wright
Maybe? The web connector might be able to do this for you. Karl On Thu, Jul 4, 2019 at 6:18 AM LIROT Daniel (Chef de projet web et collaboratif) - SG/SNUM/UNI/DETN/GPBCW/PPCW < daniel.li...@developpement-durable.gouv.fr> wrote: > Hello, > > I'd like to know if manifoldCF is able to used

Re: 'real-time'/frequent ingestion using ManifoldCF

2019-07-02 Thread Karl Wright
About the only thing I can suggest that would work within the ManifoldCF framework would be to structure your jobs so that most runs are "Minimal" runs with "Complete" runs being done every 24 hours. This should pick up documents that have been changed or added but will not go through the process

Re: JDBC Connector Max Connection Size is set as hardcoded

2019-06-28 Thread Karl Wright
Hi Cihad, The "connections" that the tab is referring to are ManifoldCF connections, not JDBC connection pool sizes, which is something completely different. The JDBC connector shares access to JDBC connections across a hard-wired pool. The number hardcoded is fine until you have more than 30

[jira] [Resolved] (CONNECTORS-1613) Array Index Out of Bounds exception, JDBC connector with attributes

2019-06-24 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright resolved CONNECTORS-1613. - Resolution: Fixed r1862000 > Array Index Out of Bounds exception, JDBC connec

Re: Unexpected job status encountered

2019-06-24 Thread Karl Wright
Created and resolved CONNECTORS-1613. Karl On Mon, Jun 24, 2019 at 8:28 AM Karl Wright wrote: > Hi Cihad, > > The unexpected job status error I cannot help you with; somehow your > database has gotten corrupted. But I'm looking into the AIOOBE issue now. > > Karl > >

[jira] [Created] (CONNECTORS-1613) Array Index Out of Bounds exception, JDBC connector with attributes

2019-06-24 Thread Karl Wright (JIRA)
Karl Wright created CONNECTORS-1613: --- Summary: Array Index Out of Bounds exception, JDBC connector with attributes Key: CONNECTORS-1613 URL: https://issues.apache.org/jira/browse/CONNECTORS-1613

Re: Unexpected job status encountered

2019-06-24 Thread Karl Wright
tor.java:2188) >> ~[?:?] >> at >> org.apache.manifoldcf.crawler.connectors.jdbc.JDBCConnector.processDocuments(JDBCConnector.java:785) >> ~[?:?] >> at >> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) >> [mcf-pull-agent.jar:?] >

[jira] [Resolved] (CONNECTORS-1519) CLIENTPROTOCOLEXCEPTION is thrown with 2.10 -> ES 6.x.y

2019-06-24 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright resolved CONNECTORS-1519. - Resolution: Fixed r1861998 Thank you, [~glaenen]! > CLIENTPROTOCOLEXCEPT

[jira] [Commented] (CONNECTORS-1519) CLIENTPROTOCOLEXCEPTION is thrown with 2.10 -> ES 6.x.y

2019-06-24 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16871104#comment-16871104 ] Karl Wright commented on CONNECTORS-1519: - Thank you for the patch! I will integrate shortly

Re: Unexpected job status encountered

2019-06-23 Thread Karl Wright
Hi Cihad, Do you have a stack trace of the ArrayIndexOutOfBounds exception? It would have to be taken from early when it started happening. What the "Error: Unexpected job status encountered: 1" error means is that the character that is stored in the job status field is not one that ManifoldCF

Re: Manifold Crawler Crashes

2019-06-20 Thread Karl Wright
this, how to achieve this. > > Also do we have to reduce some number of maximum connections in both > Repository and Output connections. can this be the symptom for heavy memory > load(due to multiple jobs running all together) that causes HEAP:-OUT OF > MEMORY. > > > > >

Re: Manifold Crawler Crashes

2019-06-20 Thread Karl Wright
feContentHandler.java:288) > at > org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:284) > at > org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146) > at > org.apache.tika.sax.xpath.MatchingContentHandler.characters(MatchingContentHandler

Re: Manifold Crawler Crashes

2019-06-20 Thread Karl Wright
> once i will restart the container through docker MCF get to load again. > > Thanks > Priya > > On Thu, Jun 20, 2019 at 3:08 PM Karl Wright wrote: > >> Please describe what you mean by "crash". What actually happens? >> >> Karl >> >> On

Re: Manifold Crawler Crashes

2019-06-20 Thread Karl Wright
Please describe what you mean by "crash". What actually happens? Karl On Thu, Jun 20, 2019, 2:04 AM Priya Arora wrote: > > > Hi, > > I am running multiple jobs(2,3) simultaneously on Manifold server and the > configuration is > > 1) For Crawler server - 16 GB RAM and 8-Core Intel(R) Xeon(R)

[jira] [Commented] (CONNECTORS-1612) Postpone files in SMBException

2019-06-19 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867466#comment-16867466 ] Karl Wright commented on CONNECTORS-1612: - I do not want to add yet more configuration

[jira] [Resolved] (CONNECTORS-1612) Postpone files in SMBException

2019-06-18 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright resolved CONNECTORS-1612. - Resolution: Fixed Fix Version/s: ManifoldCF 2.14 r1861582 > Postpone fi

[jira] [Commented] (CONNECTORS-1612) Postpone files in SMBException

2019-06-18 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866714#comment-16866714 ] Karl Wright commented on CONNECTORS-1612: - {quote} 3. If it fails, the job moves

[jira] [Assigned] (CONNECTORS-1612) Postpone files in SMBException

2019-06-18 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright reassigned CONNECTORS-1612: --- Assignee: Karl Wright > Postpone files in SMBExcept

Re: ssh connector

2019-06-17 Thread Karl Wright
Ssh is a connection technology, not a repository, so I really cannot answer this question. Karl On Mon, Jun 17, 2019 at 5:37 AM SAUNIER Maxence wrote: > Hello Karl, > > > > Do ou have any news for this question ? > > > > Thanks you, > > > > *De :* SAUNIER Maxence > *Envoyé :* mercredi 12 juin

Re: Crawling SharePoint Data

2019-06-11 Thread Karl Wright
> Furkan KAMACI > > On Wed, Jun 12, 2019 at 12:43 AM Karl Wright wrote: > >> Hi Furkan, >> >> The plugin has been necessary for crawling, period, since SharePoint >> 2010, because the native SharePoint Lists service does not fully function. >> >>

Re: Crawling SharePoint Data

2019-06-11 Thread Karl Wright
Hi Furkan, The plugin has been necessary for crawling, period, since SharePoint 2010, because the native SharePoint Lists service does not fully function. Thanks, Karl On Tue, Jun 11, 2019 at 5:30 PM Furkan KAMACI wrote: > Hi, > > One should install a plugin to crawl data from SharePoint.

Re: Error: Unexpected jobqueue status - record id X, expecting active status, saw 4 (MySQL compatible Database)

2019-06-07 Thread Karl Wright
And yes, we'd also need to hand the mysql folks a similar test case. Karl On Sat, Jun 8, 2019 at 1:53 AM Karl Wright wrote: > Here's an explanation for Postgresql about what is supposed to happen in > this case. See slide 7. > > https://www.postgresql.org/files/developer/con

Re: Error: Unexpected jobqueue status - record id X, expecting active status, saw 4 (MySQL compatible Database)

2019-06-06 Thread Karl Wright
> > Le 13 févr. 2019 à 13:58, Markus Schuch > a écrit : > > > > Hi Karl, > > > > we set the diagnostigs logger to level debug. > > > > I will get back when the error occurs again. > > > > Cheers, > > Markus > > > ------ > > &g

Re: Alternative approaches for jobs aborting on problematic docs

2019-06-06 Thread Karl Wright
taining metadata with non ASCII characters > (errors occured with chinese/japanese chars). The error mentioned a HTTP > bad request header, so most propably a 4xx/5xx HTTP error. > > Do you think we can work out something to postpone/skip these classes of > errors ? Would be great

Re: Alternative approaches for jobs aborting on problematic docs

2019-06-05 Thread Karl Wright
Please let me note that there are *tons* of errors you can get when crawling, from database errors to out-of-memory conditions to the actual ones you care about, namely errors accessing the repository. It is crucial that the connector code separate these errors into those that are fatal, those

Re: Web connector empty session cookie cache

2019-06-03 Thread Karl Wright
Hi Julien, When the session-based web crawl detects entry into a login sequence, the session cookies are cleared at that point. Essentially your symptom means that you haven't been complete about setting up your login sequence. If you make it detect the case when the session cookie is wrong,

Re: mxt file with TikaExtractor

2019-05-29 Thread Karl Wright
Hi Maxence, This should be something that you report to the Tika team. It's not something ManifoldCF can do anything about. Thanks, Karl On Wed, May 29, 2019 at 6:23 PM SAUNIER Maxence wrote: > > Hello Karl, > > > > We just realized that the TikaExtractor does not keep the line breaks for >

Re: Long running queries on jobqueue

2019-05-28 Thread Karl Wright
Oh, and you might want to check the JDBC driver to be sure it's rated as compatible with the version of the POstgresql database you are using. I imagine that can matter too. Karl On Tue, May 28, 2019 at 9:35 AM Karl Wright wrote: > When it fails again, I expect that the diagnost

Re: Long running queries on jobqueue

2019-05-28 Thread Karl Wright
t; applied the following Postgres properties : > > > > max_connections = 405 > > shared_buffers = 1024MB > > checkpoint_timeout = 900s > > max_wal_size = 14GB > > autovacuum = off > > > > Julien > > > > *De : *Karl Wright >

Re: Long running queries on jobqueue

2019-05-28 Thread Karl Wright
Hi Julien, "Error : Unexpected jobqueue status - record id 15588697113928, expecting active status, saw 0" As you said, the only thing that can be done here is to turn on diagnostic logging. Essentially, the status returned is not possible if the database is truly honoring transactional

[jira] [Resolved] (CONNECTORS-1609) SharePoint connector ignore 503 errors

2019-05-28 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright resolved CONNECTORS-1609. - Resolution: Fixed > SharePoint connector ignore 503 err

[jira] [Commented] (CONNECTORS-1610) handle error 500 in WindowsShare repository connector

2019-05-28 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16849560#comment-16849560 ] Karl Wright commented on CONNECTORS-1610: - ManifoldCF retries based on what the connector

[jira] [Resolved] (CONNECTORS-1610) handle error 500 in WindowsShare repository connector

2019-05-28 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright resolved CONNECTORS-1610. - Resolution: Won't Fix > handle error 500 in WindowsShare repository connec

[jira] [Updated] (CONNECTORS-1609) SharePoint connector ignore 503 errors

2019-05-27 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright updated CONNECTORS-1609: Attachment: CONNECTORS-1609.diff > SharePoint connector ignore 503 err

[jira] [Updated] (CONNECTORS-1609) SharePoint connector ignore 503 errors

2019-05-27 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright updated CONNECTORS-1609: Attachment: CONNECTORS-1609.diff > SharePoint connector ignore 503 err

[jira] [Commented] (CONNECTORS-1609) SharePoint connector ignore 503 errors

2019-05-27 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16848719#comment-16848719 ] Karl Wright commented on CONNECTORS-1609: - Patched attached. Please try and tell me whether

[jira] [Comment Edited] (CONNECTORS-1609) SharePoint connector ignore 503 errors

2019-05-27 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16848713#comment-16848713 ] Karl Wright edited comment on CONNECTORS-1609 at 5/27/19 8:20 AM

[jira] [Commented] (CONNECTORS-1609) SharePoint connector ignore 503 errors

2019-05-27 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16848713#comment-16848713 ] Karl Wright commented on CONNECTORS-1609: - As discussed in email, 403 actually means

[jira] [Assigned] (CONNECTORS-1609) SharePoint connector ignore 503 errors

2019-05-27 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright reassigned CONNECTORS-1609: --- Assignee: Karl Wright > SharePoint connector ignore 503 err

Re: Repository connector for source with with delta API

2019-05-24 Thread Karl Wright
> where I can store the delta token state? Or does my connector have to > create its own db table to store this? > > Regards, > Raman > > On Fri, May 24, 2019 at 6:18 PM Karl Wright wrote: > > > > So MODEL_ADD_CHANGE does not work for you, eh? > > > > You were

Re: Repository connector for source with with delta API

2019-05-24 Thread Karl Wright
se seed document ids. > > I do note that the queue shows documents 100, 110, and 120 in state > "Waiting for processing", and nothing I do seems to affect that. The > database update in JobQueue.updateExistingRecordInitial is a no-op for > these docs, as the status of them

Re: Repository connector for source with with delta API

2019-05-24 Thread Karl Wright
that model. If MODEL_ADD_CHANGE mostly works for you, then the next step is to figure out why MODEL_ADD_CHANGE_DELETE is failing. Karl On Fri, May 24, 2019 at 5:06 PM Raman Gupta wrote: > On Fri, May 24, 2019 at 4:41 PM Karl Wright wrote: > > > > For ADD_CHANGE_DELET

Re: Repository connector for source with with delta API

2019-05-24 Thread Karl Wright
For ADD_CHANGE_DELETE, the contract for addSeedDocuments() basically says that you have to include *at least* the documents that were changed, added, or deleted since the previous stamp, and if no stamp is provided, it should return ALL specified documents. Are you doing that? If you are, the

Re: SharePoint connector behavior

2019-05-24 Thread Karl Wright
Sure, you can create a ticket for that. Karl On Fri, May 24, 2019 at 11:13 AM Julien wrote: > Concerning the issue with the 403 errors, you mean bad creds for the > SharePoint server itself ? > > So I can create a ticket for at least the 503 error ? > > De : Karl Wright > E

[jira] [Resolved] (CONNECTORS-1607) SharePoint ADFS cannot connect to the sharepoint connecter

2019-05-23 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright resolved CONNECTORS-1607. - Resolution: Won't Fix Kerberos will not be supported by the SharePoint connector

[jira] [Commented] (CONNECTORS-1607) SharePoint ADFS cannot connect to the sharepoint connecter

2019-05-23 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847078#comment-16847078 ] Karl Wright commented on CONNECTORS-1607: - This is not a bug, so I will be closing

[jira] [Resolved] (CONNECTORS-1593) Memory issue on org.apache.fontbox.ttf.GlyphSubstitutionTable.readLangSysTable(GlyphSubstitutionTable.java:147)

2019-05-23 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright resolved CONNECTORS-1593. - Resolution: Not A Problem Wasn't a ManifoldCF problem, but rather a corrupt

[jira] [Resolved] (CONNECTORS-1606) Issue related to job run & throttling behaviour

2019-05-22 Thread Karl Wright (JIRA)
[ https://issues.apache.org/jira/browse/CONNECTORS-1606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright resolved CONNECTORS-1606. - Resolution: Not A Problem Please post this question to the user group rather than

Re: It is possible to dump MCF configuration/status via the REST API?

2019-05-15 Thread Karl Wright
Hi James, I'm sorry to say there isn't any integration with now-ubiquitous monitoring software in ManifoldCF. Proposals are welcome. As for getting a record of configuration -- as you know, the REST API allows you access to all the database-resident structures in MCF, which includes connections

<    4   5   6   7   8   9   10   11   12   13   >