Re: Does the ES pluging work for ES 5.5.x?

2017-09-05 Thread Steph van Schalkwyk
: ‎03/‎09/‎2017 04:24 > To: user@manifoldcf.apache.org > Subject: Re: Does the ES pluging work for ES 5.5.x? > > I've set up a project for an es-5.5 plugin and done what I could without > delving into the changes that were made to the API. You can check it out > at: > > https://sv

Re: Question about ManifoldCF 2.8

2017-09-05 Thread Karl Wright
Hi Othman, Thanks for doing the evaluation of the problem. Generally, the ManifoldCF project does not have the expertise to diagnose problems with external systems like Solr or Elasticsearch. So going to another newsgroup for those kinds of issues would be a good idea. Thanks! Karl On Tue,

Re: Question about ManifoldCF 2.8

2017-09-04 Thread Beelz Ryuzaki
Hi Karl, I'm sorry to bother on your holiday. I will try to analyze it today and let it you know what I have found. Enjoy your day ! Best regards, Othman BELHAJ. On Mon, 4 Sep 2017 at 16:06, Karl Wright wrote: > Hi Othman, > > I won't be able to look at this today; it is

Re: Question about ManifoldCF 2.8

2017-09-04 Thread Karl Wright
Hi Othman, I won't be able to look at this today; it is a holiday here. But, the "socket write" error is coming from ElasticSearch. If ES is configured to not accept documents greater than a certain size, that might explain it. Maybe the ES logs would help? I'm afraid you're going to need to

RE: Does the ES pluging work for ES 5.5.x?

2017-09-03 Thread S
Thanks Karl. I started last night. Will add ny changes. S -Original Message- From: "Karl Wright" <daddy...@gmail.com> Sent: ‎03/‎09/‎2017 04:24 To: "user@manifoldcf.apache.org" <user@manifoldcf.apache.org> Subject: Re: Does the ES pluging work for

Re: Does the ES pluging work for ES 5.5.x?

2017-09-02 Thread Karl Wright
Hi Steph, The version of ManifoldCF doesn't matter. The ManifoldCF Plugin for ES 2.0 was coded to compile against ES 2.0. It's pretty easy to see if it compiles against 5.5 -- you just change a version in the plugin's pom and rebuild. Having said that, I have no idea what APIs in ES may have

Re: Question about ManifoldCF 2.8

2017-09-01 Thread Karl Wright
(1) I would create a ticket for the "*word*" exclusion. It would be helpful to include a screen shot of the view page of your job as well. (2) I will be uploading a new ManifoldCF 2.8.1 RC shortly. Karl On Fri, Sep 1, 2017 at 12:05 PM, Beelz Ryuzaki wrote: > Hi Karl, >

Re: Question about ManifoldCF 2.8

2017-09-01 Thread Karl Wright
Hi Othman, I will respin a new 2.8.1 (RC1) to address the zookeeper issue. The failure you are seeing is "NoSuchMethodError". Therefore, the class is being found, but it is the *wrong* class. When you deployed the new release, did you deploy it in a new directory, or did you overwrite the

Re: Question about ManifoldCF 2.8

2017-09-01 Thread Karl Wright
Hi Othman, You do not need a new database instance. You can download MCF 2.8.1 RC0 from here: https://dist.apache.org/repos/dist/dev/manifoldcf/apache-manifoldcf-2.8.1 Karl On Fri, Sep 1, 2017 at 5:42 AM, Beelz Ryuzaki wrote: > Hi Karl, > > Thank you very much for your

Re: Question about ManifoldCF 2.8

2017-09-01 Thread Beelz Ryuzaki
Hi Karl, Thank you very much for your help, I'm going to try out the zookeeper example. Should I initialize a new database? And how can I run the zookeeper start-agent ? Othman. On Fri, 1 Sep 2017 at 11:37, Karl Wright wrote: > Hi Othman, > > These exceptions are now

Re: Question about ManifoldCF 2.8

2017-09-01 Thread Karl Wright
Hi Othman, These exceptions are now coming from file locking and are due to permissions problems. I suggest you go to Zookeeper for file locking. I am building a 2.8.1 release candidate. When it available for download, I'll send you the URL. Thanks, Karl On Fri, Sep 1, 2017 at 5:27 AM,

Re: Question about ManifoldCF 2.8

2017-08-31 Thread Beelz Ryuzaki
Hi Karl, By 'other place', do you mean the \lib repository? If that so, then I have already tried it and it didn't work. Othman. On Thu, 31 Aug 2017 at 18:07, Karl Wright wrote: > Hi Othman, > > I used the java dependency inspector to see what the issue is and it turns >

Re: Question about ManifoldCF 2.8

2017-08-31 Thread Beelz Ryuzaki
All the dependencies you mentioned have already been added in the options.env.win file in the multiprocess-file-example repository. On Thu, 31 Aug 2017 at 17:33, Beelz Ryuzaki wrote: > Yes, I added it in the options.env.win file. Should it be the one in the >

Re: Question about ManifoldCF 2.8

2017-08-31 Thread Karl Wright
These are the five jars that dependency analysis said should be needed: // both poi-ooxml and poi-ooxml-schemas Don't do any other jars than these, but DO make sure all four jars are moved. Thanks! Karl On Thu, Aug 31, 2017 at 11:30 AM,

Re: Question about ManifoldCF 2.8

2017-08-31 Thread Beelz Ryuzaki
Could it be a problem of elasticsearch's version ? I'm actually using 2.1.0 which is pretty old for this new version of ManifoldCF? Othman. On Thu, 31 Aug 2017 at 17:23, Beelz Ryuzaki wrote: > I moved back both the jars you mentioned and a different is showing. You > will

Re: Question about ManifoldCF 2.8

2017-08-31 Thread Karl Wright
I've looked at the dependencies; you should not have moved poi-3.15.jar. Please move that back, and commons-collections4-4.1.jar too. You *will* need to move curvesapi-1.04.jar though. Thanks, Karl On Thu, Aug 31, 2017 at 11:04 AM, Karl Wright wrote: > If you include

Re: Question about ManifoldCF 2.8

2017-08-31 Thread Karl Wright
If you include poi.jar, then all dependencies of poi.jar must also be included. This would mean that curvesapi-1.04.jar and commons-collections4-4.1.jar should also be included. Karl On Thu, Aug 31, 2017 at 10:23 AM, Beelz Ryuzaki wrote: > Hi Karl, > > I added the two

Re: Question about ManifoldCF 2.8

2017-08-31 Thread Beelz Ryuzaki
And concerning the path tabs, I will use the Unix/Windows wildcards. I think it will be enough. Othman. On Thu, 31 Aug 2017 at 16:23, Beelz Ryuzaki wrote: > Hi Karl, > > I added the two jars that you have mentioned and another one : > poi-3.15.jar . Unfortunately, there is

Re: Question about ManifoldCF 2.8

2017-08-31 Thread Karl Wright
Hi Othman, Yes, this shows that the jar we moved calls back into another jar, which will also need to be moved. *That* jar has yet another dependency too. The list of jars is thus extended to include: poi-ooxml-3.15.jar dom4j-1.6.1.jar Karl On Thu, Aug 31, 2017 at 9:25 AM, Beelz Ryuzaki

Re: Question about ManifoldCF 2.8

2017-08-31 Thread Karl Wright
Once again, I need a stack trace to diagnose what the problem is. Thanks, Karl On Thu, Aug 31, 2017 at 9:14 AM, Beelz Ryuzaki wrote: > Oh, actually it didn't solve the problem. I looked into the log file and > saw the following error: > > Error tossed :

Re: Question about ManifoldCF 2.8

2017-08-31 Thread Beelz Ryuzaki
Oh, actually it didn't solve the problem. I looked into the log file and saw the following error: Error tossed : org/apache/poi/POIXMLTypeLoader java.lang.NoClassDefFoundError: org/apache/poi/POIXMLTypeLoader. Maybe another jar is missing ? Othman. On Thu, 31 Aug 2017 at 15:01, Beelz Ryuzaki

Re: Question about ManifoldCF 2.8

2017-08-31 Thread Beelz Ryuzaki
Ok, I will try it right away and let you know if it works. Othman. On Thu, 31 Aug 2017 at 14:15, Karl Wright wrote: > Oh, and you also may need to edit your options.env files to include them > in the classpath for startup. > > Karl > > > On Thu, Aug 31, 2017 at 7:53 AM,

Re: Question about ManifoldCF 2.8

2017-08-31 Thread Karl Wright
Oh, and you also may need to edit your options.env files to include them in the classpath for startup. Karl On Thu, Aug 31, 2017 at 7:53 AM, Karl Wright wrote: > If you are amenable, there is another workaround you could try. > Specifically: > > (1) Shut down all MCF

Re: Question about ManifoldCF 2.8

2017-08-31 Thread Karl Wright
If you are amenable, there is another workaround you could try. Specifically: (1) Shut down all MCF processes. (2) Move the following two files from connector-common-lib to lib: xmlbeans-2.6.0.jar poi-ooxml-schemas-3.15.jar (3) Restart everything and see if your crawl resumes. Please let me

Re: Question about ManifoldCF 2.8

2017-08-31 Thread Karl Wright
I created a ticket for this: CONNECTORS-1450. One simple workaround is to use the external Tika server transformer rather than the embedded Tika Extractor. I'm still looking into why the jar is not being found. Karl On Thu, Aug 31, 2017 at 7:08 AM, Beelz Ryuzaki wrote:

Re: Question about ManifoldCF 2.8

2017-08-31 Thread Karl Wright
Hi Othman, The way you restrict documents with the windows share connector is by specifying information on the "Paths" tab in jobs that crawl windows shares. There is end-user documentation both online and distributed with all binary distributions that describe how to do this. Have you found

Re: Question about ManifoldCF 2.8

2017-08-31 Thread Karl Wright
I need the complete stack trace please. Are you building ManifoldCF yourself, or are you using the distributed binary? Karl On Thu, Aug 31, 2017 at 5:48 AM, Beelz Ryuzaki wrote: > I have also encountered the following problem while indexing documents in > the windows

Re: Question about ManifoldCF 2.8

2017-08-31 Thread Beelz Ryuzaki
Hello Karl, Thank you for your response, I will start using zookeeper and I will let you know if it works. I have another question to ask. Actually, I need to make some filters while crawling. I don't want to crawl some files and some folders. Could you give me an example of how to use the regex.

Re: Question about ManifoldCF 2.8

2017-08-30 Thread Steph van Schalkwyk
Thanks Karl.

Re: Question about ManifoldCF 2.8

2017-08-30 Thread Furkan KAMACI
Hi Steph, Zookeeper is a coordination service for distributed systems. Having a quorum means that more than half of the number of nodes are up and running. This is for protection of brain splitting issue. Zookeeper is a distributed system and it may be down at any time. Brain splitting can be

Re: Question about ManifoldCF 2.8

2017-08-30 Thread Karl Wright
Hi Steph, You can configure your zookeeper however you like; there is a sample configuration file included with MCF that works out of the box. But yes, we do recommend a quorum count of 3 or more. Karl On Wed, Aug 30, 2017 at 2:19 PM, Steph van Schalkwyk wrote: > Karl, > Is

Re: Question about ManifoldCF 2.8

2017-08-30 Thread Steph van Schalkwyk
Karl, Is there a requirement for the number of ZK for MCF? I've used ZK with SOLR, and the minimum quorum count is 3. Thanks Steph

Re: Question about ManifoldCF 2.8

2017-08-30 Thread Beelz Ryuzaki
I'm actually not using zookeeper. i want to know how is zookeeper different from file based sync? I also need a guidance on how to manage my pc's memory. How many Go should I allocate for the start-agent of ManifoldCF? Is 4Go enough in order to crawler 35K files ? Othman. On Wed, 30 Aug 2017 at

Re: Question about ManifoldCF 2.8

2017-08-30 Thread Karl Wright
Your disk is not writable for some reason, and that's interfering with ManifoldCF 2.8 locking. I would suggest two things: (1) Use Zookeeper for sync instead of file-based sync. (2) Have a look if you still get failures after that. Thanks, Karl On Wed, Aug 30, 2017 at 9:37 AM, Beelz Ryuzaki

Re: Question about ManifoldCF 2.8

2017-08-30 Thread Beelz Ryuzaki
Hi Mr Karl, Thank you Mr Karl for your quick response. I have looked into the ManifoldCF log file and extracted the following warnings : - Attempt to set file lock 'D:\\apache_manifoldcf-2.8\multiprocess-file-example\.\.\synch area\569\352\lock-_POOLTARGET_OUTPUTCONNECTORPOOL_ES (Lowercase)

Re: Question about ManifoldCF 2.8

2017-08-30 Thread Karl Wright
Hi Othman, ManifoldCF aborts a job if there's an error that looks like it might go away on retry, but does not. It can be either on the repository side or on the output side. If you look at the Simple History in the UI, or at the manifoldcf.log file, you should be able to get a better sense of

Re: Alfresco webscript connection problem

2017-08-22 Thread Luis Cabaceira
Hi, i'm currently onsite with customer but i can take a look and try to reproduce the issue next week. Luis On 22 August 2017 at 12:12, Maurizio Pillitu wrote: > Hi Aurélien, > > also adding Luis Cabaceira (from Alfresco Consultancy team) that can > probably help and try to

Re: Alfresco webscript connection problem

2017-08-22 Thread Maurizio Pillitu
Hi Aurélien, also adding Luis Cabaceira (from Alfresco Consultancy team) that can probably help and try to reproduce the issue. I haven't tested the connector yet against 5.2.0 Community, I'll give it a try and see if I can reproduce the issue. In the meantime, can you please confirm that you

Re: Alfresco webscript connection problem

2017-08-22 Thread Karl Wright
Hi Maurizio and Rafa, do you have any response? Karl On Wed, Aug 9, 2017 at 1:24 PM, Karl Wright wrote: > It might be the case. I'm cc'ing the resident Alfresco experts about this > now. > > Karl > > > On Wed, Aug 9, 2017 at 1:17 PM, Aurélien MAZOYER < >

Re: Documentum job stops on error

2017-07-17 Thread Karl Wright
] > *Sent:* Friday, July 14, 2017 5:41 PM > > *To:* user@manifoldcf.apache.org > *Cc:* Sharnel Merdeck Pereira; Sundarapandian Arumaidurai Vethasigamani > *Subject:* Re: Documentum job stops on error > > > > Ok, I've attached and committed an additional patch. Please let

RE: Documentum job stops on error

2017-07-17 Thread Tamizh Kumaran Thamizharasan
idurai Vethasigamani Subject: Re: Documentum job stops on error Ok, I've attached and committed an additional patch. Please let me know. Karl On Fri, Jul 14, 2017 at 7:54 AM, Tamizh Kumaran Thamizharasan <tthamizhara...@worldbankgroup.org<mailto:tthamizhara...@worldbankgroup.org>> wrote: H

Re: Documentum job stops on error

2017-07-14 Thread Karl Wright
> at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > > at java.lang.Thread.run(Thread.java:745) > > > > Regards, > > Tamizh Kumaran Thamizharasan > > > > *From:* Karl Wright [mailto:daddy...@gmail.com] > *Sen

RE: Documentum job stops on error

2017-07-14 Thread Tamizh Kumaran Thamizharasan
zh Kumaran Thamizharasan From: Karl Wright [mailto:daddy...@gmail.com] Sent: Friday, July 14, 2017 4:32 PM To: user@manifoldcf.apache.org Cc: Sharnel Merdeck Pereira; Sundarapandian Arumaidurai Vethasigamani Subject: Re: Documentum job stops on error I have created a ticket (CONNECTORS-1444) to track

RE: Documentum job stops on error

2017-07-14 Thread Tamizh Kumaran Thamizharasan
2:36 PM To: user@manifoldcf.apache.org Cc: Sharnel Merdeck Pereira; Sundarapandian Arumaidurai Vethasigamani Subject: Re: Documentum job stops on error Hi Tamizh, For any repository errors, ManifoldCF needs to know the following: (1) Is it likely to go away or not on a retry; (2) Does it substant

Re: Documentum job stops on error

2017-07-14 Thread Karl Wright
Hi Tamizh, For any repository errors, ManifoldCF needs to know the following: (1) Is it likely to go away or not on a retry; (2) Does it substantially impact the ability of ManifoldCF to properly process the document; (3) Is it generally acceptable to skip ALL documents where the error occurs.

Re: ldap authentication with crawler ui

2017-07-13 Thread Theodor Carp
ache.org> <user@manifoldcf.apache.org> Date: 13 July 2017 at 14:17:14 To: user@manifoldcf.apache.org <user@manifoldcf.apache.org> <user@manifoldcf.apache.org> Subject: Re: ldap authentication with crawler ui I wish I was familiar enough with the code for this feature that I could be > o

Re: ldap authentication with crawler ui

2017-07-12 Thread Karl Wright
Have any users out there made use of LDAP crawler-UI authentication? If so, can you have a look at Theodor's configuration and setup? Karl On Wed, Jul 12, 2017 at 10:07 AM, Theodor Carp wrote: > Hi, > > Using the below settings: > >

RE: ManifoldCF slow documentum indexing performance

2017-07-12 Thread Tamizh Kumaran Thamizharasan
Cc: Sharnel Merdeck Pereira; Sundarapandian Arumaidurai Vethasigamani Subject: Re: ManifoldCF slow documentum indexing performance Hi Tamizh, Set Xmx and Xms to same values for a better performance. Kind Regards, Furkan KAMACI On Thu, Jul 6, 2017 at 9:10 AM, Karl Wright <daddy...@gmail.

Re: ManifoldCF slow documentum indexing performance

2017-07-06 Thread Karl Wright
> Tamizh Kumaran > > > > *From:* Karl Wright [mailto:daddy...@gmail.com] > *Sent:* Wednesday, July 05, 2017 6:19 PM > *To:* user@manifoldcf.apache.org > *Cc:* Sharnel Merdeck Pereira; Sundarapandian Arumaidurai Vethasigamani > *Subject:* Re: ManifoldCF slow documentum index

RE: ManifoldCF slow documentum indexing performance

2017-07-05 Thread Tamizh Kumaran Thamizharasan
: Sharnel Merdeck Pereira; Sundarapandian Arumaidurai Vethasigamani Subject: Re: ManifoldCF slow documentum indexing performance Hi Tamizh, The likely culprit is Documentum itself. In my experience it can be quite slow, depending on how it is configured. But you can confirm that by monitoring

Re: ManifoldCF slow documentum indexing performance

2017-07-05 Thread Karl Wright
Hi Tamizh, The likely culprit is Documentum itself. In my experience it can be quite slow, depending on how it is configured. But you can confirm that by monitoring the CPU usage of Postgresql, the agents process, and the documentum server process. If none of these are CPU bound, then

Re: Sharepoint Repository Connector: Metadata Changes not causing re-index library or list items

2017-06-30 Thread Karl Wright
s from a lookup field, that > is computed from attributes of the containing list. > Such fields do not change the modified date. > > We could re-index all list items, when the list itself is modified (by > carrying down the modified date of the list for example). But this wou

Re: ManifoldCF documentum indexing issue

2017-06-22 Thread Karl Wright
ailto:daddy...@gmail.com] > *Sent:* Wednesday, June 21, 2017 5:07 PM > > *To:* user@manifoldcf.apache.org > *Cc:* Sharnel Merdeck Pereira; Sundarapandian Arumaidurai Vethasigamani > *Subject:* Re: ManifoldCF documentum indexing issue > > > > I've attached a tentative patc

RE: ManifoldCF documentum indexing issue

2017-06-22 Thread Tamizh Kumaran Thamizharasan
Cc: Sharnel Merdeck Pereira; Sundarapandian Arumaidurai Vethasigamani Subject: Re: ManifoldCF documentum indexing issue I've attached a tentative patch to the ticket CONNECTORS-1434. Please confirm whether or not the patch works for you before I commit it to trunk. Karl On Wed, Jun 21, 2017

Re: ManifoldCF documentum indexing issue

2017-06-21 Thread Karl Wright
il.com] > *Sent:* Wednesday, June 21, 2017 3:35 PM > > *To:* user@manifoldcf.apache.org > *Cc:* Sharnel Merdeck Pereira; Sundarapandian Arumaidurai Vethasigamani > *Subject:* Re: ManifoldCF documentum indexing issue > > > > I've created a ticket, CONNECTO

RE: ManifoldCF documentum indexing issue

2017-06-21 Thread Tamizh Kumaran Thamizharasan
Vethasigamani Subject: Re: ManifoldCF documentum indexing issue I've created a ticket, CONNECTORS-1434, to look at the file name issues. Karl On Wed, Jun 21, 2017 at 5:44 AM, Karl Wright <daddy...@gmail.com<mailto:daddy...@gmail.com>> wrote: There is no good way to handle a case where

Re: ManifoldCF documentum indexing issue

2017-06-21 Thread Karl Wright
>> >> Which makes the file non readable and missing content stream error is >> thrown. >> >> >> >> If there is any work around to overcome this issue, kindly share it with >> us. >> >> >> >> Regards, >> >> Tamizh Kumaran

Re: ManifoldCF documentum indexing issue

2017-06-14 Thread Karl Wright
s that don’t have body content and keeps trying to >>> push the content to Solr after each failure, but Solr couldn’t accept the >>> content and throws the same error. >>> >>> 4) Over the time, the manifold job stops with the error thrown by >>>

Re: ManifoldCF documentum indexing issue

2017-06-14 Thread Karl Wright
>> >> >> Please let know if there is any configuration change which can help us >> resolve this issue. >> >> >> >> Please find the attached manifoldCF error log,Solr error log and agent >> log. >> >> >> >> Regards, >

Re: ManifoldCF documentum indexing issue

2017-06-14 Thread Karl Wright
ich can help us > resolve this issue. > > > > Please find the attached manifoldCF error log,Solr error log and agent log. > > > > Regards, > > Tamizh Kumaran. > > > > *From:* Karl Wright [mailto:daddy...@gmail.com] > *Sent:* Tuesday, June 13, 2017 2:

Re: ManifoldCF documentum indexing issue

2017-06-13 Thread Karl Wright
Hi Tamizh, The reported error is 'Error from server at http://localhost:8983/solr/ documentum_manifoldcf_stg: String index out of range: -188'. The message seemingly indicates that the error was *received* from the solr server for one specific document. ManifoldCF does not recognize the error

Re: UTF-8 Format from Confluence to Solr

2017-06-12 Thread Karl Wright
Committed a fix. Karl On Mon, Jun 12, 2017 at 7:27 PM, Karl Wright wrote: > There's already a ticket for this, assigned to me. CONNECTORS-1251. I'll > freshen it up. > > Karl > > > > > On Mon, Jun 12, 2017 at 2:52 PM, Furkan KAMACI > wrote: > >>

Re: UTF-8 Format from Confluence to Solr

2017-06-12 Thread Karl Wright
There's already a ticket for this, assigned to me. CONNECTORS-1251. I'll freshen it up. Karl On Mon, Jun 12, 2017 at 2:52 PM, Furkan KAMACI wrote: > Hi Marisol, > > You can create a ticket from here: https://issues.apache. > org/jira/projects/CONNECTORS > > Kind

Re: UTF-8 Format from Confluence to Solr

2017-06-12 Thread Furkan KAMACI
Hi Marisol, You can create a ticket from here: https://issues.apache.org/jira/projects/CONNECTORS Kind Regards, Furkan KAMACI 12 Haz 2017 Pzt, saat 18:25 tarihinde Marisol Redondo < marisol.redondo.gar...@gmail.com> şunu yazdı: > How can I do that? > > On 1 June 2017 at 16:43, Antonio David

Re: UTF-8 Format from Confluence to Solr

2017-06-12 Thread Marisol Redondo
How can I do that? On 1 June 2017 at 16:43, Antonio David Pérez Morales < adperezmora...@gmail.com> wrote: > Hi Marisol > > Could you mind to create a ticket and provide a patch? > > This way we can test it in our ends and include it for the next Manifold > release. > > Thanks > > Regards > >

Re: UTF-8 Format from Confluence to Solr

2017-06-01 Thread Antonio David Pérez Morales
Hi Marisol Could you mind to create a ticket and provide a patch? This way we can test it in our ends and include it for the next Manifold release. Thanks Regards 2017-06-01 16:28 GMT+02:00 Marisol Redondo : > I fixed the problem. > > The problem is that

Re: UTF-8 Format from Confluence to Solr

2017-06-01 Thread Marisol Redondo
I fixed the problem. The problem is that the Confluence connector is getting the entity of the request with the default encoding ("ISO-8859-1"), and not UTF-8. To fix that, I made a change in the Confluence connector, and each time is reading the request's entity I use

Re: ManifoldCF Indexing and Deletion

2017-05-26 Thread Karl Wright
Hi Tamizh, What do you mean by "incremental run"? If you mean what happens when you click "Start minimal" here: http://manifoldcf.apache.org/release/release-2.7.1/en_US/end-user-documentation.html#executing, then this behavior is the way it is supposed to work. You must click the "Start"

Re: ManifoldCF

2017-05-03 Thread Karl Wright
Hi Claudiu, First, it looks like you are running MCF as a single process. That is fine; if you were running a multiprocess setup you'd want to be sure to increase the memory size of all the agents processes, and not worry about any other MCF processes. Second, when you put Tika in the pipeline,

Re: Windows share connector : fetch ACL for an incremental job

2017-05-02 Thread Karl Wright
Hi Olivier, It was a long time ago that the Windows Share Connector was designed, but at the time it was determined that you could change ACLs that affected security on a document without changing the document itself, and thus the document's modified date was insufficient by itself to signal a

Re: email job is down

2017-04-28 Thread Karl Wright
Hi Cihad, The right thing to do is to capture this exception: >> Caused by: javax.mail.MessagingException: * BYE JavaMail Exception: java.io.IOException: Connection dropped by server? << ... and throw a ServiceInterruption when it is seen, instead of a ManifoldCFException. Can you

Re: Delete IDs with JDBC connector

2017-04-27 Thread julien . massiera
ulien.massi...@francelabs.com> wrote: > > Hi Karl, > > I was manually starting the job for test purpose, but even if I schedule it > with job invocation "Complete" and "Scan every document once", the missing > IDs from the database are not deleted i

Re: Delete IDs with JDBC connector

2017-04-27 Thread julien . massiera
if I schedule it > with job invocation "Complete" and "Scan every document once", the missing > IDs from the database are not deleted in my Solr index (no trace of any > 'document deletion' event in the history). > I should mention that I only use the 'Seeding que

Re: Delete IDs with JDBC connector

2017-04-26 Thread Karl Wright
history >> for that connection. >> >> Thanks, >> Karl >> >> >> >> On Wed, Apr 26, 2017 at 10:42 AM, <julien.massi...@francelabs.com> wrote: >> >>> Hi Karl, >>> >>> I was manually starting the job for test pu

Re: Delete IDs with JDBC connector

2017-04-26 Thread julien . massiera
e of any > 'document deletion' event in the history). > I should mention that I only use the 'Seeding query' and 'Data query' and I > am not using the $(STARTTIME) and $(ENDTIME) variables in my seeding query. > > Julien > > Le 26.04.2017 16:05, Karl Wright a écrit :

Re: Delete IDs with JDBC connector

2017-04-26 Thread Karl Wright
page that told MCF to "forget" all the history >> for that connection. >> >> Thanks, >> Karl >> >> >> >> On Wed, Apr 26, 2017 at 10:42 AM, <julien.massi...@francelabs.com> wrote: >> >>> Hi Karl, >>> >>> I

Re: Delete IDs with JDBC connector

2017-04-26 Thread Karl Wright
gt; I should mention that I only use the 'Seeding query' and 'Data query' and >> I am not using the $(STARTTIME) and $(ENDTIME) variables in my seeding >> query. >> >> Julien >> >> Le 26.04.2017 16:05, Karl Wright a écrit : >> >> Hi Julien, >>

Re: Delete IDs with JDBC connector

2017-04-26 Thread Karl Wright
gt; > How are you starting the job? If you use "Start minimal", deletion would > not take place. If your job is a continuous one, this is also the case. > > Thanks, > Karl > > On Wed, Apr 26, 2017 at 9:52 AM, <julien.massi...@francelabs.com> wrote: > >> Hi t

Re: Delete IDs with JDBC connector

2017-04-26 Thread Karl Wright
; > I am using MCF 2.6 with the JDBC connector to crawl an Oracle Database and > index the data into a Solr server, and it works very well. However, when I > perform a delta re-crawl, the new IDs are correctly retrieved from the > Database but those who have been deleted are not &quo

Re: Email filtering does not work for Excahange Server

2017-04-24 Thread Karl Wright
Hi Cihad, The implementation for filtering is pretty generic. Details are handled by the javax mail jar, and there's not much visibility with what it is doing. I think this is something you will need to experiment with to figure out what the issue is. It may be, for instance, that it's the

Re: ManifoldCf Documentum Negative ACL

2017-04-06 Thread Karl Wright
pA >> but NONE(1) access for User2. Hence lowest access takes precedence, User2 >> won’t have access to ACL_1. >> >> >> >> - User4 is part of Group A and has READ(3) access to ACL_1 >> >> >> >> - User 5 is part of GroupD

Re: ManifoldCf Documentum Negative ACL

2017-04-06 Thread Karl Wright
owest *r_accessor_permit *takes precedence. > > > > The query > > *select r_accessor_name, r_accessor_permit, r_is_group from dm_acl where > object_name =’’ * > > will retrieve accessor_name and permission for acl. > > > > The query > > *sele

RE: Multilingual support with manifolds

2017-03-29 Thread Konrad Holl
[mailto:sree...@gmail.com] Sent: Dienstag, 28. März 2017 17:43 To: user@manifoldcf.apache.org Subject: Re: Multilingual support with manifolds Thanks a lot for your responses. Reason for asking was that sharepoint content is in german & japanese. We would like to get the content to Solr. If I unders

Re: manifoldcf build

2017-03-28 Thread Karl Wright
Hi Cihad, There are no changes to the build process. However, there have been significant changes to the dependencies. You will need to do the following: (1) Set your JAVA_HOME to point to JDK 8. The previous requirement was JDK 7. (2) ant clean-core-deps make-core-deps (3) ant clean build

Re: Multilingual support with manifolds

2017-03-28 Thread Cihad Guzel
ant in the search engine itself > (SharePoint). It will detect the languages and apply linguistic processing > as needed during indexing and search time. > > > > -Konrad > > > > *From:* Karl Wright [mailto:daddy...@gmail.com] > *Sent:* Dienstag, 28. März 2017 13:22

RE: Multilingual support with manifolds

2017-03-28 Thread Konrad Holl
To: user@manifoldcf.apache.org Subject: Re: Multilingual support with manifolds Hi, ManifoldCF uses utf-8 and binary throughout for its actual function, so it is not language specific in any way at that level. Its UI has been localized (more or less) for four languages: English, Spanish

Re: Multilingual support with manifolds

2017-03-28 Thread Karl Wright
Hi, ManifoldCF uses utf-8 and binary throughout for its actual function, so it is not language specific in any way at that level. Its UI has been localized (more or less) for four languages: English, Spanish, Japanese, and Chinese. Hope that helps, Karl On Tue, Mar 28, 2017 at 6:13 AM,

Re: SharePoint crawler ArrayIndexOutOfBoundException in log

2017-03-17 Thread Cihad Guzel
Hi, I use oracle jdk1.8.0_77 . I will try new http client version and return to you. Thanks Cihad Güzel 2017-03-17 23:38 GMT+03:00 Markus Schuch : > Hi, > > i think this may caused by > > https://issues.apache.org/jira/browse/HTTPCLIENT-1715 > > which was fixed in

Re: SharePoint crawler ArrayIndexOutOfBoundException in log

2017-03-17 Thread Karl Wright
Hi Markus, Good catch. Yes, this could do it. I'm going to update trunk's dependencies and see if that fixes the issue. Karl On Fri, Mar 17, 2017 at 4:38 PM, Markus Schuch wrote: > Hi, > > i think this may caused by > >

Re: SharePoint crawler ArrayIndexOutOfBoundException in log

2017-03-17 Thread Markus Schuch
Hi, i think this may caused by https://issues.apache.org/jira/browse/HTTPCLIENT-1715 which was fixed in httpclient 4.5.2 There is a very similar stacktrace in https://issues.apache.org/jira/browse/HTTPCLIENT-1686 which is also linked to HTTPCLIENT-1715. Cheers, Markus Am 17.03.2017 um

Re: SharePoint crawler ArrayIndexOutOfBoundException in log

2017-03-17 Thread Cihad Guzel
No. I don't use any custom library. I try with manifoldcf trunk on my notebook. I install sharepoint 2013 on ms server 2012 for testing with default configuration. 17 Mar 2017 16:05 tarihinde "Karl Wright" yazdı: > Hmm, I can see no way this can happen. Are you by any

Re: SharePoint crawler ArrayIndexOutOfBoundException in log

2017-03-17 Thread Karl Wright
Hmm, I can see no way this can happen. Are you by any chance using a modified version of the HttpClient library? Karl On Fri, Mar 17, 2017 at 8:09 AM, Karl Wright wrote: > Hi Cihad, > > This is very interesting because the problem is coming from Httpclient's > NTLM engine.

Re: SharePoint crawler ArrayIndexOutOfBoundException in log

2017-03-17 Thread Karl Wright
Hi Cihad, This is very interesting because the problem is coming from Httpclient's NTLM engine. The allocated packet size for the Type 1 message is being exceeded, which I didn't think was even possible. This may be a result of credentials that you have supplied being strange in some way. Let

Re: The job got stuck when JDBC Connector got org.postgresql.util.PSQLException

2017-03-08 Thread Cheng Zeng
<daddy...@gmail.com> Sent: 08 March 2017 16:04 To: user@manifoldcf.apache.org Subject: Re: The job got stuck when JDBC Connector got org.postgresql.util.PSQLException Hi Cheng, The issue is that your JDBC connection is generating a version string that has a character zero (0x0) in it, and post

Re: Advice on which PostgreSQL to use with ManifoldCF 2.6

2017-03-08 Thread Karl Wright
t; important information. > > > > This Zookeeper issue happened in the middle of the night and no one would > have manually instigated it. > > > > Best Regards, > > > > Guy > > > > *From:* Karl Wright [mailto:daddy...@gmail.com] > *Sent:* 08 March 20

Re: The job got stuck when JDBC Connector got org.postgresql.util.PSQLException

2017-03-08 Thread Karl Wright
Hi Cheng, The issue is that your JDBC connection is generating a version string that has a character zero (0x0) in it, and postgresql doesn't allow that. You get to specify the version string query as part of the job definition -- can you look at that and see how you are getting this back? It

RE: Advice on which PostgreSQL to use with ManifoldCF 2.6

2017-03-08 Thread Standen Guy
in the middle of the night and no one would have manually instigated it. Best Regards, Guy From: Karl Wright [mailto:daddy...@gmail.com] Sent: 08 March 2017 13:45 To: user@manifoldcf.apache.org Subject: Re: Advice on which PostgreSQL to use with ManifoldCF 2.6 Hi Guy, If nobody recycled Zookeeper

Re: Advice on which PostgreSQL to use with ManifoldCF 2.6

2017-03-08 Thread Karl Wright
>> >> >> Best Regards, >> >> >> >> Guy >> >> >> >> *From:* Karl Wright [mailto:daddy...@gmail.com] >> *Sent:* 08 March 2017 12:32 >> >> *To:* user@manifoldcf.apache.org >> *Subject:* Re: Advice on which Pos

Re: Advice on which PostgreSQL to use with ManifoldCF 2.6

2017-03-08 Thread Karl Wright
than this. > > > > I’ll try and reproduce the problem with forensic logging on and append the > traces to connectors-1395. > > > > Best Regards, > > > > Guy > > > > *From:* Karl Wright [mailto:daddy...@gmail.com] > *Sent:* 08 March 2017 12:32 > &

RE: Advice on which PostgreSQL to use with ManifoldCF 2.6

2017-03-08 Thread Standen Guy
trace than this. I’ll try and reproduce the problem with forensic logging on and append the traces to connectors-1395. Best Regards, Guy From: Karl Wright [mailto:daddy...@gmail.com] Sent: 08 March 2017 12:32 To: user@manifoldcf.apache.org Subject: Re: Advice on which PostgreSQL to use

<    5   6   7   8   9   10   11   12   13   14   >