Hi Karl, I haven’t experienced any job aborts, so all seems OK in that respect. Is there anything I can do to reduce these errors in the first place, or it is just to be expected with the nature of the multiple worker threads and the query types issued by ManifoldCF? Best Regards,
Guy From: Karl Wright [mailto:daddy...@gmail.com] Sent: 06 August 2018 12:16 To: user@manifoldcf.apache.org Subject: Re: PostgreSQL version to support MCF v2.10 These are exactly the same kind of issue as the first "error" reported. They will be retried. If they did not get retried, they would abort the job immediately. Karl On Mon, Aug 6, 2018 at 6:57 AM Standen Guy <guy.stan...@uk.fujitsu.com<mailto:guy.stan...@uk.fujitsu.com>> wrote: Hi Karl, Thanks for the prompt response regarding the first error example. Do you have a view as to second error i.e. “2018-08-03 15:52:42.855 BST [5272] ERROR: could not serialize access due to concurrent update 2018-08-03 15:52:42.855 BST [5272] STATEMENT: SELECT id,status,checktime FROM jobqueue WHERE dochash=$1 AND jobid=$2 FOR UPDATE 2018-08-03 15:52:42.855 BST [7424] ERROR: could not serialize access due to concurrent update 2018-08-03 15:52:42.855 BST [7424] STATEMENT: SELECT id,status,checktime FROM jobqueue WHERE dochash=$1 AND jobid=$2 FOR UPDATE 2018-08-03 15:52:42.855 BST [5716] ERROR: could not serialize access due to concurrent update “ These errors don’t suggest a retry may sort them out - is this an issue? Many Thanks, Guy From: Karl Wright [mailto:daddy...@gmail.com<mailto:daddy...@gmail.com>] Sent: 06 August 2018 10:52 To: user@manifoldcf.apache.org<mailto:user@manifoldcf.apache.org> Subject: Re: PostgreSQL version to support MCF v2.10 Ah, the following errors: >>>>>> 2018-08-03 15:52:25.218 BST [4140] ERROR: could not serialize access due to read/write dependencies among transactions 2018-08-03 15:52:25.218 BST [4140] DETAIL: Reason code: Canceled on identification as a pivot, during conflict in checking. 2018-08-03 15:52:25.218 BST [4140] HINT: The transaction might succeed if retried. <<<<<< ... occur because of concurrent transactions. The transaction is indeed retried when this occurs, so unless your job aborts, you are fine. Karl On Mon, Aug 6, 2018 at 5:49 AM Karl Wright <daddy...@gmail.com<mailto:daddy...@gmail.com>> wrote: What errors are these? Please include them and I can let you know. Karl On Mon, Aug 6, 2018 at 4:50 AM Standen Guy <guy.stan...@uk.fujitsu.com<mailto:guy.stan...@uk.fujitsu.com>> wrote: Thank you Karl and Steph, Steph, yes I don’t seem to have any issues with running the MCF jobs, but am concerned about the PostgreSQL errors. Do you ( or anyone else) have a view on the errors I have seen in the PostgreSQL logs - is this something you have seen with 10.4 and if so was it corrected by changing some settings? Best Regards Guy From: Steph van Schalkwyk [mailto:st...@remcam.net<mailto:st...@remcam.net>] Sent: 03 August 2018 23:21 To: user@manifoldcf.apache.org<mailto:user@manifoldcf.apache.org> Subject: Re: PostgreSQL version to support MCF v2.10 I'm using 10.4 with no issues. One or two of the recommended settings for MCF have changed between 9.6 and 10. Simple to resolve though. Steph On Fri, Aug 3, 2018 at 1:29 PM, Karl Wright <daddy...@gmail.com<mailto:daddy...@gmail.com>> wrote: Hi Guy, I use Postgresql 9.6 myself and have found no issues with it. I don't know about v 10 however. Karl On Fri, Aug 3, 2018 at 11:32 AM Standen Guy <guy.stan...@uk.fujitsu.com<mailto:guy.stan...@uk.fujitsu.com>> wrote: Hi Karl/All, I am upgrading from MCF v2.6 supported by PostgreSQL v 9.3.16 to MCF v2.10. I wonder if there is any official advice as to which version of PostgreSQL will support MCF v2.10? The MCF v2.10 build and deployment instructions still suggest that PostgreSQL 9.3 is the latest tested version of PostgreSQL. Given that PostgreSQL 9.3.x is going end of life next month ( Sept 2018), is there a preferred newer version that should be used? As an experiment I have installed MCF 2.10 supported by PostgreSQL 10.4. From the outside all seems to work OK, but investigation of the PostgreSQL logs shows a lot of errors: e.g. “2018-08-03 15:50:00.629 BST [7920] LOG: database system was shut down at 2018-08-03 15:47:30 BST 2018-08-03 15:50:00.734 BST [6344] LOG: database system is ready to accept connections 2018-08-03 15:52:11.140 BST [6460] WARNING: there is already a transaction in progress 2018-08-03 15:52:11.219 BST [6460] WARNING: there is no transaction in progress 2018-08-03 15:52:13.844 BST [5716] WARNING: there is already a transaction in progress 2018-08-03 15:52:13.879 BST [5716] WARNING: there is no transaction in progress 2018-08-03 15:52:25.218 BST [4140] ERROR: could not serialize access due to read/write dependencies among transactions 2018-08-03 15:52:25.218 BST [4140] DETAIL: Reason code: Canceled on identification as a pivot, during conflict in checking. 2018-08-03 15:52:25.218 BST [4140] HINT: The transaction might succeed if retried. 2018-08-03 15:52:25.218 BST [4140] STATEMENT: INSERT INTO jobqueue (jobid,docpriority,checktime,docid,needpriority,dochash,id,checkaction,status) VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9) 2018-08-03 15:52:25.219 BST [5800] ERROR: could not serialize access due to read/write dependencies among transactions 2018-08-03 15:52:25.219 BST [5800] DETAIL: Reason code: Canceled on identification as a pivot, during conflict in checking. 2018-08-03 15:52:25.219 BST [5800] HINT: The transaction might succeed if retried. 2018-08-03 15:52:25.219 BST [5800] STATEMENT: INSERT INTO jobqueue (jobid,docpriority,checktime,docid,needpriority,dochash,id,checkaction,status) VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9) 2018-08-03 15:52:25.222 BST [5692] ERROR: could not serialize access due to read/write dependencies among transactions 2018-08-03 15:52:25.222 BST [5692] DETAIL: Reason code: Canceled on identification as a pivot, during conflict in checking. 2018-08-03 15:52:25.222 BST [5692] HINT: The transaction might succeed if retried. 2018-08-03 15:52:25.222 BST [5692] STATEMENT: INSERT INTO jobqueue (jobid,docpriority,checktime,docid,needpriority,dochash,id,checkaction,status) VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9) 2018-08-03 15:52:28.149 BST [4140] ERROR: could not serialize access due to read/write dependencies among transactions 2018-08-03 15:52:28.149 BST [4140] DETAIL: Reason code: Canceled on identification as a pivot, during write. 2018-08-03 15:52:28.149 BST [4140] HINT: The transaction might succeed if retried. 2018-08-03 15:52:28.149 BST [4140] STATEMENT: UPDATE intrinsiclink SET processid=$1,isnew=$2 WHERE jobid=$3 AND parentidhash=$4 AND linktype=$5 AND childidhash=$6 2018-08-03 15:52:28.261 BST [5156] ERROR: could not serialize access due to read/write dependencies among transactions 2018-08-03 15:52:28.261 BST [5156] DETAIL: Reason code: Canceled on identification as a pivot, during write. 2018-08-03 15:52:28.261 BST [5156] HINT: The transaction might succeed if retried.” And “2018-08-03 15:52:42.855 BST [5272] ERROR: could not serialize access due to concurrent update 2018-08-03 15:52:42.855 BST [5272] STATEMENT: SELECT id,status,checktime FROM jobqueue WHERE dochash=$1 AND jobid=$2 FOR UPDATE 2018-08-03 15:52:42.855 BST [7424] ERROR: could not serialize access due to concurrent update 2018-08-03 15:52:42.855 BST [7424] STATEMENT: SELECT id,status,checktime FROM jobqueue WHERE dochash=$1 AND jobid=$2 FOR UPDATE 2018-08-03 15:52:42.855 BST [5716] ERROR: could not serialize access due to concurrent update 2018-08-03 15:52:42.855 BST [5716] STATEMENT: SELECT id,status,checktime FROM jobqueue WHERE dochash=$1 AND jobid=$2 FOR UPDATE 2018-08-03 15:52:42.856 BST [1328] ERROR: could not serialize access due to concurrent update 2018-08-03 15:52:42.856 BST [1328] STATEMENT: SELECT id,status,checktime FROM jobqueue WHERE dochash=$1 AND jobid=$2 FOR UPDATE 2018-08-03 15:52:42.856 BST [5800] ERROR: could not serialize access due to concurrent update 2018-08-03 15:52:42.856 BST [5800] STATEMENT: SELECT id,status,checktime FROM jobqueue WHERE dochash=$1 AND jobid=$2 FOR UPDATE” Do you have any advice as to whether it is sensible to use PostgreSQL v10.x and if so can these errors be overcome? Best Regards, Guy Unless otherwise stated, this email has been sent from Fujitsu Services Limited (registered in England No 96056); Fujitsu EMEA PLC (registered in England No 2216100) both with registered offices at: 22 Baker Street, London W1U 3BW<https://maps.google.com/?q=22+Baker+Street,+London+W1U+3BW&entry=gmail&source=g>; PFU (EMEA) Limited, (registered in England No 1578652) and Fujitsu Laboratories of Europe Limited (registered in England No. 4153469) both with registered offices at: Hayes Park Central, Hayes End Road, Hayes, Middlesex, UB4 8FE. This email is only for the use of its intended recipient. Its contents are subject to a duty of confidence and may be privileged. Fujitsu does not guarantee that this email has not been intercepted and amended or that it is virus-free. Unless otherwise stated, this email has been sent from Fujitsu Services Limited (registered in England No 96056); Fujitsu EMEA PLC (registered in England No 2216100) both with registered offices at: 22 Baker Street, London W1U 3BW; PFU (EMEA) Limited, (registered in England No 1578652) and Fujitsu Laboratories of Europe Limited (registered in England No. 4153469) both with registered offices at: Hayes Park Central, Hayes End Road, Hayes, Middlesex, UB4 8FE. This email is only for the use of its intended recipient. Its contents are subject to a duty of confidence and may be privileged. Fujitsu does not guarantee that this email has not been intercepted and amended or that it is virus-free. Unless otherwise stated, this email has been sent from Fujitsu Services Limited (registered in England No 96056); Fujitsu EMEA PLC (registered in England No 2216100) both with registered offices at: 22 Baker Street, London W1U 3BW; PFU (EMEA) Limited, (registered in England No 1578652) and Fujitsu Laboratories of Europe Limited (registered in England No. 4153469) both with registered offices at: Hayes Park Central, Hayes End Road, Hayes, Middlesex, UB4 8FE. This email is only for the use of its intended recipient. Its contents are subject to a duty of confidence and may be privileged. Fujitsu does not guarantee that this email has not been intercepted and amended or that it is virus-free. Unless otherwise stated, this email has been sent from Fujitsu Services Limited (registered in England No 96056); Fujitsu EMEA PLC (registered in England No 2216100) both with registered offices at: 22 Baker Street, London W1U 3BW; PFU (EMEA) Limited, (registered in England No 1578652) and Fujitsu Laboratories of Europe Limited (registered in England No. 4153469) both with registered offices at: Hayes Park Central, Hayes End Road, Hayes, Middlesex, UB4 8FE. This email is only for the use of its intended recipient. Its contents are subject to a duty of confidence and may be privileged. Fujitsu does not guarantee that this email has not been intercepted and amended or that it is virus-free.
