Re: Job error during WindowsShare repository connector indexation

This error: >> WARN 2017-10-09 08:23:56,284 (Idle cleanup thread) - MCF|MCF-agent|apache.manifoldcf.lock|Attempt to set file lock 'mcf/mcf_home/./syncharea/551/442/lock-_POOLTARGET__ REPOSITORYCONNECTORPOOL_SmbFileShare.lock' failed: No such file or directory java.io.IOException: No such file

Re: Best practices for Postgresql configuration

Hi Olivier, We've tried versions of Postgresql beyond 9.3, and they seem to work, but there's always a possibility that the query plans will turn out badly. But this is unlikely. The automatic vacuum operation in Postgresql has gotten much better over time. You do not need to pause MCF to do

Re: How to extract text content and index in elastic-search

Hi Dileepa, MCF passes content through its processing chain as binary. It's up to the output connection configuration to decide if the output should be rendered as text or binary, and it is there that a different decision would need to be made. IIRC there's a flag you can set that chooses

Re: Alfresco webscript connection problem

ain with the correct version and set the >> following properties: >> >> >> >> >> >> 5.2.f >> >> 5.0 >> >> >> >> before running a “mvn clean install”. However, I can see that the >> alfr

Re: MCF 2.8.1 agent logs

gging.connectors.debug("Testing DEBUG on logging.xml settings by Luis > Cabaceira"); > > That is not outputted in manifoldcf.log. To prove this i've executed the > same line in error and it does get written. > > Logging.connectors.error("Testing Error log on logging.xml settin

Re: MCF 2.8.1 agent logs

Hi Julien, Thanks for bringing the documentation issue to our attention. Can you create a ticket for that? As for the problem: Do you not see log output for manifoldcf.log for (say) the unaltered single-process example? It has been a while since the port to log4j2 was done but I'm pretty sure

Re: Question about ManifoldCF 2.8

ded in the documentation. > > > Many thanks, > > Othman BELHAJ > > On Mon, 18 Sep 2017 at 12:15, Karl Wright <daddy...@gmail.com> wrote: > >> Hi Othman, >> >> What you do is add an attribute through the adjuster. Then, in Solr or >> Elastic Sear

Re: Question about ManifoldCF 2.8

Hello Karl, > > I'm interested in knowing if there is a way to tag the indexed documents > with ManifoldCF ? > > Many thanks, > > Othman BELHAJ > > On Fri, 8 Sep 2017 at 21:43, Karl Wright <daddy...@gmail.com> wrote: > >> Hi Othman, >> >> There are tw

Re: Problem with JSON output of MCF web api

The reason for the failure is likely because we had to move off of simple json to a different library due to Apache withdrawing support for simple json's license. Tests passed but clearly we must have missed something. Karl On Tue, Sep 12, 2017 at 6:26 AM, Karl Wright <daddy...@gmail.

Re: Problem with JSON output of MCF web api

Hi Adrian, Can you create a ticket and include this stack trace? Thanks! Karl On Tue, Sep 12, 2017 at 6:23 AM, Adrian Conlon wrote: > Hi List, > > > > I’m attempting to upgrade my manifoldcf installation scripts from v2.5 to > v2.8.1 (bit of a jump, I know!). > >

Re: Question about ManifoldCF 2.8

m> wrote: >> >>> Thank you, Karl. I will try to combine Postgresql with zookeeper and let >>> you know. >>> >>> Othman. >>> >>> On Wed, 6 Sep 2017 at 13:18, Karl Wright <daddy...@gmail.com> wrote: >>> >>>> No, y

Re: Does the ES pluging work for ES 5.5.x?

;https://mail.google.com/mail/u/0/#> > <http://linkedin.com/in/vanschalkwyk> > > On Wed, Sep 6, 2017 at 12:31 PM, Karl Wright <daddy...@gmail.com> wrote: > >> Do you want me to find out who at ES might be able to assist you? I >> still have some con

Re: Does the ES pluging work for ES 5.5.x?

Do you want me to find out who at ES might be able to assist you? I still have some contacts there. Kalr On Wed, Sep 6, 2017 at 1:30 PM, Karl Wright <daddy...@gmail.com> wrote: > A guy by the name of Bartlomiej Superson. > > Karl > > > On Wed, Sep 6, 2017 at 1:20 PM,

Re: Does the ES pluging work for ES 5.5.x?

t;http://www.remcam.net/> Skype: svanschalkwyk >> <https://mail.google.com/mail/u/0/#> >> <http://linkedin.com/in/vanschalkwyk> >> >> On Wed, Sep 6, 2017 at 11:38 AM, Karl Wright <daddy...@gmail.com> wrote: >> >>> Hopefully you can submit new

Re: Does the ES pluging work for ES 5.5.x?

+1.314.452. <+1+314+452+2896>2896st...@remcam.net http://remcam.net > <http://www.remcam.net/> Skype: svanschalkwyk > <https://mail.google.com/mail/u/0/#> > <http://linkedin.com/in/vanschalkwyk> > > On Wed, Sep 6, 2017 at 10:59 AM, Karl Wright <daddy...@

Re: Does the ES pluging work for ES 5.5.x?

2. <+1+314+452+2896>2896st...@remcam.net http://remcam.net > <http://www.remcam.net/> Skype: svanschalkwyk > <https://mail.google.com/mail/u/0/#> > <http://linkedin.com/in/vanschalkwyk> > > On Wed, Sep 6, 2017 at 10:42 AM, Karl Wright <daddy...@gmail.com> wrote

Re: Does the ES pluging work for ES 5.5.x?

If you submit a patch against the San directory I created and attach it to the ticket, I will commit it. Karl On Sep 6, 2017 11:33 AM, "Steph van Schalkwyk" wrote: > Karl, > Anywhere I could shelve my code? I 'm stuck at >

Re: Question about ManifoldCF 2.8

On Wed, 6 Sep 2017 at 12:56, Karl Wright <daddy...@gmail.com> wrote: > >> Hi Othman, >> >> HSQLDB stores all tables in memory so you need to size it accordingly. >> That is one reason we prefer Postgresql for production deployments. >> >> Thanks, >> Ka

Re: Does the ES pluging work for ES 5.5.x?

t; >> *Steph van Schalkwyk* >> Principal, Remcam Search Engines >> +1.314.452. <+1+314+452+2896>2896st...@remcam.net http://remcam.net >> <http://www.remcam.net/> Skype: svanschalkwyk >> <https://mail.google.com/mail/u/0/#> >> <http://linkedin.

Re: Does the ES pluging work for ES 5.5.x?

M, S <st...@remcam.net> wrote: > >> Thanks Karl. >> I started last night. Will add ny changes. >> S >> -- >> From: Karl Wright <daddy...@gmail.com> >> Sent: ‎03/‎09/‎2017 04:24 >> To: user@manifoldcf.apache.org >

Re: Question about ManifoldCF 2.8

i Karl, >> >> I'm sorry to bother on your holiday. I will try to analyze it today and >> let it you know what I have found. Enjoy your day ! >> >> Best regards, >> >> Othman BELHAJ. >> >> On Mon, 4 Sep 2017 at 16:06, Karl Wright <daddy...@gmail.com&

Re: Question about ManifoldCF 2.8

e one error which is bugging me. It is a socket > write error. You will find attached the simple history report. > Surprisingly, I didn't have any stack trace in the ManifoldCF log file. > > Best regards, > > Othman. > > On Fri, 1 Sep 2017 at 19:39, Karl Wright <daddy.

Re: Does the ES pluging work for ES 5.5.x?

Hi Steph, The version of ManifoldCF doesn't matter. The ManifoldCF Plugin for ES 2.0 was coded to compile against ES 2.0. It's pretty easy to see if it compiles against 5.5 -- you just change a version in the plugin's pom and rebuild. Having said that, I have no idea what APIs in ES may have

Re: Question about ManifoldCF 2.8

ow can I solve this issue, please? > > Thank you very much, have a nice week-end, > > Othman > On Fri, 1 Sep 2017 at 16:46, Karl Wright <daddy...@gmail.com> wrote: > >> Hi Othman, >> >> I will respin a new 2.8.1 (RC1) to address the zookeeper iss

Re: Question about ManifoldCF 2.8

normally, but in the second I got a new stack trace concerning the > POI. Moreover, the runzookeeper.bat doesn't run properly. It shows me the > stack trace attached. > > Ps: > The second attached file contains the POI stack trace. > > Othman. > > On Fri, 1 Sep 2017 at

Re: Question about ManifoldCF 2.8

y much for your help, I'm going to try out the zookeeper > example. Should I initialize a new database? And how can I run the > zookeeper start-agent ? > > Othman. > > On Fri, 1 Sep 2017 at 11:37, Karl Wright <daddy...@gmail.com> wrote: > >> Hi Othman, >>

Re: Question about ManifoldCF 2.8

ady to use the zookeeper example. > Could you guide through it? I don't know if I follow the same steps in the > file based example, I may not get stack traces. > > Thanks, > Othman > > On Thu, 31 Aug 2017 at 18:19, Karl Wright <daddy...@gmail.com> wrote: > >>

Re: Question about ManifoldCF 2.8

, Karl Wright <daddy...@gmail.com> wrote: > It's not related at all to elasticsearch. > Karl > > > On Thu, Aug 31, 2017 at 11:26 AM, Beelz Ryuzaki <i93oth...@gmail.com> > wrote: > >> Could it be a problem of elasticsearch's version ? I'm actually using >> 2

Re: Question about ManifoldCF 2.8

I've looked at the dependencies; you should not have moved poi-3.15.jar. Please move that back, and commons-collections4-4.1.jar too. You *will* need to move curvesapi-1.04.jar though. Thanks, Karl On Thu, Aug 31, 2017 at 11:04 AM, Karl Wright <daddy...@gmail.com> wrote: > If yo

Re: Question about ManifoldCF 2.8

I added the two jars that you have mentioned and another one : > poi-3.15.jar . Unfortunately, there is another error showing. This time, it > concerns excel files. You will find attached the stack trace. > > Othman. > > On Thu, 31 Aug 2017 at 15:32, Karl Wright <daddy...@gmail.c

Re: Question about ManifoldCF 2.8

t; > On Thu, 31 Aug 2017 at 15:16, Karl Wright <daddy...@gmail.com> wrote: > >> Once again, I need a stack trace to diagnose what the problem is. >> >> Thanks, >> Karl >> >> >> On Thu, Aug 31, 2017 at 9:14 AM, Beelz Ryuzaki <i93oth...@gmai

Re: Question about ManifoldCF 2.8

Beelz Ryuzaki <i93oth...@gmail.com> wrote: >> >>> Ok, I will try it right away and let you know if it works. >>> >>> Othman. >>> >>> On Thu, 31 Aug 2017 at 14:15, Karl Wright <daddy...@gmail.com> wrote: >>> >>>&g

Re: Question about ManifoldCF 2.8

Oh, and you also may need to edit your options.env files to include them in the classpath for startup. Karl On Thu, Aug 31, 2017 at 7:53 AM, Karl Wright <daddy...@gmail.com> wrote: > If you are amenable, there is another workaround you could try. > Specifically: > > (1)

Re: Question about ManifoldCF 2.8

know what happens. Karl On Thu, Aug 31, 2017 at 7:33 AM, Karl Wright <daddy...@gmail.com> wrote: > I created a ticket for this: CONNECTORS-1450. > > One simple workaround is to use the external Tika server transformer > rather than the embedded Tika Extractor. I'm stil

Re: Question about ManifoldCF 2.8

wrote: > Yes, I'm actually using the latest binary version, and my job got stuck on > that specific file. > The job status is still Running. You can see it in the attached file. For > your information, the job started yesterday. > > Thanks, > > Othman > > On Thu, 31 A

Re: Question about ManifoldCF 2.8

filters while crawling. I don't want to crawl some files and some > folders. Could you give me an example of how to use the regex. Does the > regex allow to use /i to ignore cases ? > > Thanks, > Othman > > On Wed, 30 Aug 2017 at 19:53, Karl Wright <daddy...@gmail.com>

Re: Question about ManifoldCF 2.8

n example of how to use the regex. Does the >> regex allow to use /i to ignore cases ? >> >> Thanks, >> Othman >> >> On Wed, 30 Aug 2017 at 19:53, Karl Wright <daddy...@gmail.com> wrote: >> >>> Hi Beelz, >>> >>> File-based s

Re: Question about ManifoldCF 2.8

Hi Steph, You can configure your zookeeper however you like; there is a sample configuration file included with MCF that works out of the box. But yes, we do recommend a quorum count of 3 or more. Karl On Wed, Aug 30, 2017 at 2:19 PM, Steph van Schalkwyk wrote: > Karl, > Is

Re: Question about ManifoldCF 2.8

synapses being the elasticsearch output connection. > Moreover, the job uses Tika to extract metadata and a file system as a > repository connection. During the job, I don't extract the content of the > documents. I was wandering if the issue comes from elasticsearch ? > >

Re: Question about ManifoldCF 2.8

Hi Othman, ManifoldCF aborts a job if there's an error that looks like it might go away on retry, but does not. It can be either on the repository side or on the output side. If you look at the Simple History in the UI, or at the manifoldcf.log file, you should be able to get a better sense of

Re: Alfresco webscript connection problem

Hi Maurizio and Rafa, do you have any response? Karl On Wed, Aug 9, 2017 at 1:24 PM, Karl Wright <daddy...@gmail.com> wrote: > It might be the case. I'm cc'ing the resident Alfresco experts about this > now. > > Karl > > > On Wed, Aug 9, 2017 at 1:17 PM, Aurélie

[ANNOUNCE] ManifoldCF 2.8 has been released

This release includes a new connector (Nuxeo) as well as numerous fixes and improvements to other connectors. Solr 6.x support and ElasticSearch 5.x support have also been added. Please join me in congratulating the ManifoldCF team and the ManifoldCF contributors for their invaluable assistance

Re: Documentum job stops on error

0and%20DocumentumException.java> > file change on https://issues.apache.org/jira/secure/attachment/ > 12877277/CONNECTORS-1444.patch should be sufficient. > > > > Regards, > > Tamizh Kumaran Thamizharasan > > > > *From:* Karl Wright [mailto:daddy...@gmail.com

Re: Documentum job stops on error

83) > > at java.security.AccessController.doPrivileged(Native Method) > > at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run( > TCPTransport.java:682) > > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) >

Re: Documentum job stops on error

Hi Tamizh, For any repository errors, ManifoldCF needs to know the following: (1) Is it likely to go away or not on a retry; (2) Does it substantially impact the ability of ManifoldCF to properly process the document; (3) Is it generally acceptable to skip ALL documents where the error occurs.

Re: ldap authentication with crawler ui

Have any users out there made use of LDAP crawler-UI authentication? If so, can you have a look at Theodor's configuration and setup? Karl On Wed, Jul 12, 2017 at 10:07 AM, Theodor Carp wrote: > Hi, > > Using the below settings: > >

Re: ManifoldCF slow documentum indexing performance

documentum server run script, java heap is having value as below. > > *-Xmx512m -Xms32m* > > > > Is there any way to speed up the indexing through heap configuration or > increasing hardware? > > If so, Kindly share us the details. > > > > Regards, >

Re: ManifoldCF slow documentum indexing performance

Hi Tamizh, The likely culprit is Documentum itself. In my experience it can be quite slow, depending on how it is configured. But you can confirm that by monitoring the CPU usage of Postgresql, the agents process, and the documentum server process. If none of these are CPU bound, then

Re: Sharepoint Repository Connector: Metadata Changes not causing re-index library or list items

If it's computed from other attributes, then don't the other attributes need to change in order for the lookup attribute's value to change? Karl On Fri, Jun 30, 2017 at 9:13 AM, wrote: > Hi Karl, > > we found out, that the affected metadate comes from a lookup

Re: ManifoldCF documentum indexing issue

org> wrote: > Thanks Karl. > > > > After installing the patch, filename with double quotes and backslashes > were getting indexed to Solr and the issue is resolved. > > > > Regards, > > Tamizh Kumaran Thamizharasan > > > > *From:* Karl Wright [m

Re: ManifoldCF documentum indexing issue

lse > > > > On starting the job with above configuration, we are getting “missing > content stream” . > > Please find the attached file for complete log trace. > > > > Regards, > > Tamizh Kumaran Thamizharasan > > > > *From:* Karl Wright [mailto:daddy...@gma

Re: ManifoldCF documentum indexing issue

I've created a ticket, CONNECTORS-1434, to look at the file name issues. Karl On Wed, Jun 21, 2017 at 5:44 AM, Karl Wright <daddy...@gmail.com> wrote: > There is no good way to handle a case where Solr doesn't like the file > name. About the only thing that could be done would

Re: ManifoldCF documentum indexing issue

instance involved. I'm also quite concerned that considerations of backwards compatibility may have been lost at some point with Solr, since heretofore I could count on older versions of SolrJ working with newer versions of Solr. Please clarify what the current policy is.... Thanks, Karl <<<<&

Re: ManifoldCF documentum indexing issue

I posted the pertinent question to the solr dev list. Let's see what they say. Thanks, Karl On Wed, Jun 14, 2017 at 9:04 AM, Karl Wright <daddy...@gmail.com> wrote: > Hi, > > The exception in the solr.log should be reported as a Solr bug. It is not > emanating from the Ti

Re: ManifoldCF documentum indexing issue

ich can help us > resolve this issue. > > > > Please find the attached manifoldCF error log,Solr error log and agent log. > > > > Regards, > > Tamizh Kumaran. > > > > *From:* Karl Wright [mailto:daddy...@gmail.com] > *Sent:* Tuesday, June 13, 2017 2:

Re: ManifoldCF documentum indexing issue

Hi Tamizh, The reported error is 'Error from server at http://localhost:8983/solr/ documentum_manifoldcf_stg: String index out of range: -188'. The message seemingly indicates that the error was *received* from the solr server for one specific document. ManifoldCF does not recognize the error

Re: UTF-8 Format from Confluence to Solr

Committed a fix. Karl On Mon, Jun 12, 2017 at 7:27 PM, Karl Wright <daddy...@gmail.com> wrote: > There's already a ticket for this, assigned to me. CONNECTORS-1251. I'll > freshen it up. > > Karl > > > > > On Mon, Jun 12, 2017 at 2:52 PM, Furkan KAMACI

Re: UTF-8 Format from Confluence to Solr

There's already a ticket for this, assigned to me. CONNECTORS-1251. I'll freshen it up. Karl On Mon, Jun 12, 2017 at 2:52 PM, Furkan KAMACI wrote: > Hi Marisol, > > You can create a ticket from here: https://issues.apache. > org/jira/projects/CONNECTORS > > Kind

Re: ManifoldCF Indexing and Deletion

Hi Tamizh, What do you mean by "incremental run"? If you mean what happens when you click "Start minimal" here: http://manifoldcf.apache.org/release/release-2.7.1/en_US/end-user-documentation.html#executing, then this behavior is the way it is supposed to work. You must click the "Start"

[ANNOUNCE] Apache ManifoldCF 2.7.1 has shipped

must ensure that your browser cache is flushed before the fix for this problem will be in effect. Thanks! Karl Wright

Re: ManifoldCF

Hi Claudiu, First, it looks like you are running MCF as a single process. That is fine; if you were running a multiprocess setup you'd want to be sure to increase the memory size of all the agents processes, and not worry about any other MCF processes. Second, when you put Tika in the pipeline,

Re: Windows share connector : fetch ACL for an incremental job

Hi Olivier, It was a long time ago that the Windows Share Connector was designed, but at the time it was determined that you could change ACLs that affected security on a document without changing the document itself, and thus the document's modified date was insufficient by itself to signal a

Re: email job is down

Hi Cihad, The right thing to do is to capture this exception: >> Caused by: javax.mail.MessagingException: * BYE JavaMail Exception: java.io.IOException: Connection dropped by server? << ... and throw a ServiceInterruption when it is seen, instead of a ManifoldCFException. Can you

Re: Delete IDs with JDBC connector

ien > > Le 26.04.2017 17:20, Karl Wright a écrit : > > Oh, never mind. I see the issue, which is that without the version query, > documents that don't appear in the result list *at all* are never removed > from the map. I'll create a ticket. > > Karl > > >

Re: Delete IDs with JDBC connector

CONNECTORS-1419. Karl On Wed, Apr 26, 2017 at 11:20 AM, Karl Wright <daddy...@gmail.com> wrote: > Oh, never mind. I see the issue, which is that without the version query, > documents that don't appear in the result list *at all* are never removed > from the map. I'll create a t

Re: Delete IDs with JDBC connector

Oh, never mind. I see the issue, which is that without the version query, documents that don't appear in the result list *at all* are never removed from the map. I'll create a ticket. Karl On Wed, Apr 26, 2017 at 11:10 AM, Karl Wright <daddy...@gmail.com> wrote: > Hi Julien, >

Re: Delete IDs with JDBC connector

om> wrote: > Hi Karl, > > I was manually starting the job for test purpose, but even if I schedule > it with job invocation "Complete" and "Scan every document once", the > missing IDs from the database are not deleted in my Solr index (no trace of > any 'document

Re: Delete IDs with JDBC connector

Hi Julien, How are you starting the job? If you use "Start minimal", deletion would not take place. If your job is a continuous one, this is also the case. Thanks, Karl On Wed, Apr 26, 2017 at 9:52 AM, wrote: > Hi the MCF community, > > I am using MCF 2.6

Re: Email filtering does not work for Excahange Server

Hi Cihad, The implementation for filtering is pretty generic. Details are handled by the javax mail jar, and there's not much visibility with what it is doing. I think this is something you will need to experiment with to figure out what the issue is. It may be, for instance, that it's the

Re: ManifoldCf Documentum Negative ACL

Hi Sharnel, I've attached a patch to the CONNECTORS-1401 ticket. Please let me know if it works for you. Thanks, Karl On Thu, Apr 6, 2017 at 5:52 PM, Karl Wright <daddy...@gmail.com> wrote: > Hi Sharnel, > > I've created CONNECTORS-1401 to track this issue; I will try to get

Re: ManifoldCf Documentum Negative ACL

owest *r_accessor_permit *takes precedence. > > > > The query > > *select r_accessor_name, r_accessor_permit, r_is_group from dm_acl where > object_name =’’ * > > will retrieve accessor_name and permission for acl. > > > > The query > > *sele

Re: manifoldcf build

Hi Cihad, There are no changes to the build process. However, there have been significant changes to the dependencies. You will need to do the following: (1) Set your JAVA_HOME to point to JDK 8. The previous requirement was JDK 7. (2) ant clean-core-deps make-core-deps (3) ant clean build

Re: Multilingual support with manifolds

Hi, ManifoldCF uses utf-8 and binary throughout for its actual function, so it is not language specific in any way at that level. Its UI has been localized (more or less) for four languages: English, Spanish, Japanese, and Chinese. Hope that helps, Karl On Tue, Mar 28, 2017 at 6:13 AM,

Re: SharePoint crawler ArrayIndexOutOfBoundException in log

ira/browse/HTTPCLIENT-1715 > > which was fixed in httpclient 4.5.2 > > There is a very similar stacktrace in > > https://issues.apache.org/jira/browse/HTTPCLIENT-1686 > > which is also linked to HTTPCLIENT-1715. > > Cheers, > Markus > &

Re: SharePoint crawler ArrayIndexOutOfBoundException in log

Hmm, I can see no way this can happen. Are you by any chance using a modified version of the HttpClient library? Karl On Fri, Mar 17, 2017 at 8:09 AM, Karl Wright <daddy...@gmail.com> wrote: > Hi Cihad, > > This is very interesting because the problem is coming from Httpclient'

Re: SharePoint crawler ArrayIndexOutOfBoundException in log

Hi Cihad, This is very interesting because the problem is coming from Httpclient's NTLM engine. The allocated packet size for the Type 1 message is being exceeded, which I didn't think was even possible. This may be a result of credentials that you have supplied being strange in some way. Let

Re: Advice on which PostgreSQL to use with ManifoldCF 2.6

t; important information. > > > > This Zookeeper issue happened in the middle of the night and no one would > have manually instigated it. > > > > Best Regards, > > > > Guy > > > > *From:* Karl Wright [mailto:daddy...@gmail.com] > *Sent:* 08 March 20

Re: The job got stuck when JDBC Connector got org.postgresql.util.PSQLException

Hi Cheng, The issue is that your JDBC connection is generating a version string that has a character zero (0x0) in it, and postgresql doesn't allow that. You get to specify the version string query as part of the job definition -- can you look at that and see how you are getting this back? It

Re: Advice on which PostgreSQL to use with ManifoldCF 2.6

2017 at 8:37 AM, Karl Wright <daddy...@gmail.com> wrote: > Right, sorry, I overlooked this attachment in your original mail. Have a > look at the ticket for updated status of the research, or later posts in > this thread. > > Karl > > > On Wed, Mar 8, 2017 at 8:06 AM,

Re: Advice on which PostgreSQL to use with ManifoldCF 2.6

than this. > > > > I’ll try and reproduce the problem with forensic logging on and append the > traces to connectors-1395. > > > > Best Regards, > > > > Guy > > > > *From:* Karl Wright [mailto:daddy...@gmail.com] > *Sent:* 08 March 2017 12:32 > &

Re: Advice on which PostgreSQL to use with ManifoldCF 2.6

) Get a thread dump (2) Get a snapshot of the log at that point (3) Shut down the agents process and the UI process (4) Start up the agents process and the UI process You should *not* need to recycle Zookeeper, ever. Thanks, Karl On Wed, Mar 8, 2017 at 8:16 AM, Karl Wright <daddy...@gmail.

Re: Advice on which PostgreSQL to use with ManifoldCF 2.6

Hi Guy, The agents thread dump shows that there's a lock stuck from somewhere; I expect it's from the UI. Next time this happens, could you get a thread dump for the UI process as well as from the agents process? Thanks!! Karl On Wed, Mar 8, 2017 at 6:12 AM, Karl Wright <daddy...@gmail.

Re: Advice on which PostgreSQL to use with ManifoldCF 2.6

2.6 > e.g. PostgreSQL 9.3.16 or PostgreSQL 9.6.2? > > 2) For a production system on a single server running a single MCF agents > process would you recommend the file based synchronisation locking or > zookeeper based synchronisation locking. With the file based > synchronisa

Re: MS Exhange support

Hi Cihad, I've been able to connect to Exchange in the past; you need to use IMAP if I recall correctly. Karl On Sun, Mar 5, 2017 at 11:53 AM, Cihad Guzel wrote: > Hi, > > Does MCF Email connector support Microsoft Exchange? It doesn't support as > much as I can see. > >

Re: Request-URI Too Long Error

Hi Furkan, The error is coming from Solr. How is your Solr connection configured? If you are using /update/extract, your documents should be sent via POST, not GET. Karl On Thu, Mar 2, 2017 at 8:24 AM, Furkan KAMACI wrote: > Hi, > > When I test E-mail connector I

Re: Metadata adjuster

ing MCF 2.4, that does *not* have the SolrJ 6.x version you will need to work with Solr 6.x. That may well be where the trouble lies. Please upgrade to MCF 2.6 to rule out that possibility. If that does not fix the issue, then I will bring one of our resident Solr experts into the conversation. Thanks, Karl

Re: Metadata adjuster

Ah, sorry once again. It is definitely the update/extract handler in the log entry you sent. I am quite busy at the moment and will review this evening further. Thanks, Karl On Wed, Feb 22, 2017 at 11:21 AM, Karl Wright <daddy...@gmail.com> wrote: > Hi Marisol, > > The [INFO

Re: Metadata adjuster

Hi Marisol, The [INFO] log statement you sent earlier was not an /update/extract request, and your Solr connection is set up to send to the Solr Cell /update/extract endpoint. Can you look again in your logs and find the *right* [INFO] statement? Thanks!! Karl On Wed, Feb 22, 2017 at 10:52

Re: Metadata adjuster

Ah, never mind -- I need you instead to view the Solr connection, and paste that in an email. Basically, I want to be sure you are not inadvertantly disabling metadata to Solr. Thanks, Karl On Wed, Feb 22, 2017 at 10:39 AM, Karl Wright <daddy...@gmail.com> wrote: > This is how

Re: Metadata adjuster

false and too true, > but I'll take your advice and set to true. > > I don't know why you can't see it, but it's the 4 stage > > On 22 February 2017 at 15:26, Karl Wright <daddy...@gmail.com> wrote: > >> Hi Marisol, >> >> Some observations. >> (1) It mak

Re: Metadata adjuster

tLuceneDocument( >> AddUpdateCommand.java:82) > > at org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate( >> DirectUpdateHandler2.java:277) > > at org.apache.solr.update.DirectUpdateHandler2.addDoc0( >> DirectUpdateHandler2.java:211) > > > > Thanks > > &g

Re: Additional information from external database

e database > - retrieve the file number > - add it to a certain field > > I do know little to nothing about java, but I am able to teach myself if > necessary. Is there any starting point to begin with developing my on > transformation connector? > > Thanks in advan

Re: Metadata adjuster

Hi Marisol, Can you find the [INFO] entry in the Solr log for this document? That should help clear up any confusion. Also, for what it is worth, MCF 1.10 is not using a SolrJ that is up to date with Solr 6.x. That could be the source of the problem Is there any reason you are using a 1.x

Re: extract email attachment

activities.deleteDocument(documentIdentifier); >> continue; >> } >> >> I updated these lines: (lines :1485 and 1586) >> int index2 = di.indexOf("/", index1 + 1); >> as like: >> int index2 = di

CONNECTORS-1372 -- loss of more metadata fields when Reader-type metadata is used

Hi all, Just found another bad bug that results in the loss of metadata fields and other bizarre effects. This occurs when metadata fields of type Reader or Date are used. The issue is conversion of the Reader or Date to a string winds up corrupting an iterator over the metadata collection.

Re: extract email attachment

Here's the full code for this class: https://svn.apache.org/repos/asf/manifoldcf/trunk/connectors/email/connector/src/main/java/org/apache/manifoldcf/crawler/connectors/email/EmailConnector.java Karl On Tue, Feb 7, 2017 at 5:14 PM, Karl Wright <daddy...@gmail.com> wrote: >

Re: extract email attachment

= attachmentIndex; ... } Karl On Tue, Feb 7, 2017 at 4:43 PM, Cihad Guzel <cguz...@gmail.com> wrote: > Hi Karl, > > I added LOG line for testing. It looks attachmentIndex is null. > > 2017-02-08 0:11 GMT+03:00 Karl Wright <daddy...@gmail.com>: > >> I atta

Re: extract email attachment

Correction: the only metadata attribute we set is the attachment(s) mimetype (as a multivalued field) -- this doesn't currently include the attachment data. Karl On Tue, Feb 7, 2017 at 1:14 PM, Karl Wright <daddy...@gmail.com> wrote: > Hi Cihad, > > The email connect

Re: extract email attachment

Hi Cihad, The email connector is providing the attachment data unextracted to the output connector as metadata attribute data. There are no transformation connectors that look at this metadata. Solr cell also probably does not handle binary in random metadata attributes the proper way. The

Re: PKIX error, when using https URL in RSS Connection

Hi Joachim, The RSS connector by default should use "trust everything", which is why there's no selection for that in the UI. The code clearly has support for this in place. The only way it would not work is if the https connection you are trying to set up requires public key authentication, or

<    1   2   3   4   5   6   7   8   9   10   >