Re: Advice on which PostgreSQL to use with ManifoldCF 2.6

I responded in the ticket. Let's continue our conversation there. It seems to me that we're seeing a catastrophic communication failure between Zookeeper and the MCF processes. I have no idea why that happened. Are these on the same machine, or different machines? If different machines, it

RE: Advice on which PostgreSQL to use with ManifoldCF 2.6

Hi Karl, I have attached a file to CONNECTORS-1395 which shows an excerpt from the Zookeeper console log. Unfortunately I don’t have enough history in the console to see much before the event and may have lost some important information. This Zookeeper issue happened in the

Re: Advice on which PostgreSQL to use with ManifoldCF 2.6

Hi Guy, If nobody recycled Zookeeper intentionally at 00:26:00, is it possible that some automated process "cleaned up" zookeeper temporary files out from under it at around this time? Really, we're seeing a catastrophic failure of Zookeeper to retain anything beyond that point, and I have no

Re: Advice on which PostgreSQL to use with ManifoldCF 2.6

Right, sorry, I overlooked this attachment in your original mail. Have a look at the ticket for updated status of the research, or later posts in this thread. Karl On Wed, Mar 8, 2017 at 8:06 AM, Standen Guy wrote: > Hi Karl, > > Attached is the MCF trace that

RE: Advice on which PostgreSQL to use with ManifoldCF 2.6

Hi Karl, Attached is the MCF trace that includes all the logging pertaining to the FATAL error ( the last entry is at 00:36:56,798 there was no further logging until 08:00 when I looked at the jobs in the morning). This is the same trace I sent on the original mail, I’m afraid I have no more

Re: Advice on which PostgreSQL to use with ManifoldCF 2.6

Hi Guy, It looks like the lock unhappiness in the log may be due to you recycling Zookeeper while the MCF agents process was still running. This isn't a good idea and will cause carnage of this kind. So I have to conclude at the moment that the lock problems seem to be either due to a deadlock

Re: Advice on which PostgreSQL to use with ManifoldCF 2.6

Hi Guy, The agents thread dump shows that there's a lock stuck from somewhere; I expect it's from the UI. Next time this happens, could you get a thread dump for the UI process as well as from the agents process? Thanks!! Karl On Wed, Mar 8, 2017 at 6:12 AM, Karl Wright

Re: Advice on which PostgreSQL to use with ManifoldCF 2.6

Hi Guy, See https://issues.apache.org/jira/browse/CONNECTORS-590. When you see "unexpected" this is not a good sign: >> “ERROR 2017-03-08 00:25:30,433 (Worker thread '14') - Exception tossed: Unexpected jobqueue status - record id 1488898668325, expecting active status, saw 4

RE: Advice on which PostgreSQL to use with ManifoldCF 2.6

Many Thanks Karl From: Karl Wright [mailto:daddy...@gmail.com] Sent: 06 March 2017 09:24 To: user@manifoldcf.apache.org Subject: Re: Advice on which PostgreSQL to use with ManifoldCF 2.6 Hi Guy, (1) I have no experience with PostgresSQL versions beyond 9.3, but I doubt you would have problems.

RE: Advice on which PostgreSQL to use with ManifoldCF 2.6

Hi Karl, Thanks for that I will try version 2.6. Whilst moving to MCF 2.6 I would potentially like to upgrade my backend PostgreSQL version from 9.3.5. 1) Do you have a recommendation for which PostgreSQL to use with MCF 2.6 e.g. PostgreSQL 9.3.16 or PostgreSQL 9.6.2? 2) For

RE: Advice on which PostgreSQL to use with ManifoldCF 2.6

Hi Karl, Thanks for coming back so quickly. Unfortunately I wasn’t using a JCIFS connection. One of the issues I was seeing was between a crawl of an intranet site (no explicit throttling other than number of connections) and scheduled crawl (every 5 mins) to a relational DB