Re: [ccp4bb] xds crashes
Sounds like your kernel might think "forkxds" is creating a "fork bomb". Too many sub-processes firing off in too short a time triggers this. On one of my systems, I fixed it by editing /etc/security/limits.d/20-nproc.conf so that users are allowed ~10x more pids than normal. This tends to prevent these mysterious "Killed" errors. HTH? -James Holton MAD Scientist On 11/28/2022 7:44 AM, Demetres D. Leonidas wrote: Dear all, I do not know if this is the right list and I would like to apologize if it is not. We have repeatedly experienced xds crashes in machines running ubuntu 22.04.1 with the following message repeated several times at the INTEGRATE step /usr/local/bin/forkxds: line 60: 3427 Done echo "$itask" 3428 Killed | $amain We are trying to process data from P13 at EMBL-Hamburg. We are running XDS version Jan 10, 2022 BUILT=20220820 Any ideas ? Demetres To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/
Re: [ccp4bb] xds crashes
Dear Demetres, I agree with what James says: this is the operating system trying to to prevent, in an over-zealous way, the forking of too many or too big (in terms of memory) processes. In other words, XDS hasn't changed, but the operating system changed - it applies different limits to its processes. Your friendly system administrator should try to find out exactly which limit(s) changed, and reset the value(s). Perhaps a hint is in the Release Notes of Ubuntu 22.04 ? Pls report here when you've found it - currently you are the only one reporting it. Best wishes, Kay On Mon, 28 Nov 2022 12:06:43 -0800, James Holton wrote: >Sounds like your kernel might think "forkxds" is creating a "fork >bomb". Too many sub-processes firing off in too short a time triggers >this. On one of my systems, I fixed it by editing >/etc/security/limits.d/20-nproc.conf so that users are allowed ~10x more >pids than normal. This tends to prevent these mysterious "Killed" errors. > >HTH? > >-James Holton >MAD Scientist > > >On 11/28/2022 7:44 AM, Demetres D. Leonidas wrote: >> Dear all, >> >> I do not know if this is the right list and I would like to apologize >> if it is not. >> >> We have repeatedly experienced xds crashes in machines running ubuntu >> 22.04.1 with the following message repeated several times at the >> INTEGRATE step >> >> /usr/local/bin/forkxds: line 60: 3427 Done echo "$itask" >> >> 3428 Killed | $amain >> >> We are trying to process data from P13 at EMBL-Hamburg. >> >> We are running XDS version Jan 10, 2022 BUILT=20220820 >> >> Any ideas ? >> >> Demetres >> > > > >To unsubscribe from the CCP4BB list, click the following link: >https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > >This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing >list hosted by www.jiscmail.ac.uk, terms & conditions are available at >https://www.jiscmail.ac.uk/policyandsecurity/ To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/
Re: [ccp4bb] xds crashes
Dear Kay, after some digging and help from Max Nanao we found that the problem was the memory. It was fulling rapidly. The workaround was to: 1. unzip the cbf files prior to processing 2. use maximum number of jobs 1 3. use number of processors 4 4. run xds_par Many thanks for all your help Demetres On 11/29/2022 9:03 AM, Kay Diederichs wrote: Dear Demetres, I agree with what James says: this is the operating system trying to to prevent, in an over-zealous way, the forking of too many or too big (in terms of memory) processes. In other words, XDS hasn't changed, but the operating system changed - it applies different limits to its processes. Your friendly system administrator should try to find out exactly which limit(s) changed, and reset the value(s). Perhaps a hint is in the Release Notes of Ubuntu 22.04 ? Pls report here when you've found it - currently you are the only one reporting it. Best wishes, Kay On Mon, 28 Nov 2022 12:06:43 -0800, James Holton wrote: Sounds like your kernel might think "forkxds" is creating a "fork bomb". Too many sub-processes firing off in too short a time triggers this. On one of my systems, I fixed it by editing /etc/security/limits.d/20-nproc.conf so that users are allowed ~10x more pids than normal. This tends to prevent these mysterious "Killed" errors. HTH? -James Holton MAD Scientist On 11/28/2022 7:44 AM, Demetres D. Leonidas wrote: Dear all, I do not know if this is the right list and I would like to apologize if it is not. We have repeatedly experienced xds crashes in machines running ubuntu 22.04.1 with the following message repeated several times at the INTEGRATE step /usr/local/bin/forkxds: line 60: 3427 Done echo "$itask" 3428 Killed | $amain We are trying to process data from P13 at EMBL-Hamburg. We are running XDS version Jan 10, 2022 BUILT=20220820 Any ideas ? Demetres To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/ To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/ -- --- Dr. Demetres D. Leonidas Professor in Biochemistry Department of Biochemistry & Biotechnology University of Thessaly Biopolis 41500 Larissa, Greece - Tel. +302410 565278 Tel. +302410 565297 (Lab) Tel. +304210 565263 (X-ray) Fax. +302410 565290 E-mail: ddleoni...@bio.uth.gr ORCID ID orcid.org/-0002-3874-2523 --- -- Αυτό το email έχει ελεγχθεί για ιούς από το Avast antivirus. www.avast.com To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/
Re: [ccp4bb] xds crashes
Dear Demetres, sounds a bit drastic! The second point alone would certainly suffice, because then no forking is done. What kind of hardware is that? I would expect that on a contemporary workstation you have 8GB of RAM or more; that would allow for 8 or more processes. Do you have a swapfile? If not, create one. Please check out https://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/Performance Best wishes, Kay On Tue, 29 Nov 2022 09:34:49 +0200, Demetres D. Leonidas wrote: >Dear Kay, > >after some digging and help from Max Nanao we found that the problem >was the memory. It was fulling rapidly. The workaround was to: > >1. unzip the cbf files prior to processing > >2. use maximum number of jobs 1 > >3. use number of processors 4 > >4. run xds_par > >Many thanks for all your help > >Demetres > >On 11/29/2022 9:03 AM, Kay Diederichs wrote: >> Dear Demetres, >> >> I agree with what James says: this is the operating system trying to to >> prevent, in an over-zealous way, the >> forking of too many or too big (in terms of memory) processes. >> In other words, XDS hasn't changed, but the operating system changed - it >> applies different limits to its processes. >> Your friendly system administrator should try to find out exactly which >> limit(s) changed, and reset the value(s). >> Perhaps a hint is in the Release Notes of Ubuntu 22.04 ? >> >> Pls report here when you've found it - currently you are the only one >> reporting it. >> >> Best wishes, >> Kay >> >> >> On Mon, 28 Nov 2022 12:06:43 -0800, James Holton wrote: >> >>> Sounds like your kernel might think "forkxds" is creating a "fork >>> bomb". Too many sub-processes firing off in too short a time triggers >>> this. On one of my systems, I fixed it by editing >>> /etc/security/limits.d/20-nproc.conf so that users are allowed ~10x more >>> pids than normal. This tends to prevent these mysterious "Killed" errors. >>> >>> HTH? >>> >>> -James Holton >>> MAD Scientist >>> >>> >>> On 11/28/2022 7:44 AM, Demetres D. Leonidas wrote: Dear all, I do not know if this is the right list and I would like to apologize if it is not. We have repeatedly experienced xds crashes in machines running ubuntu 22.04.1 with the following message repeated several times at the INTEGRATE step /usr/local/bin/forkxds: line 60: 3427 Done echo "$itask" 3428 Killed | $amain We are trying to process data from P13 at EMBL-Hamburg. We are running XDS version Jan 10, 2022 BUILT=20220820 Any ideas ? Demetres >>> >>> >>> To unsubscribe from the CCP4BB list, click the following link: >>> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 >>> >>> This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing >>> list hosted by www.jiscmail.ac.uk, terms & conditions are available at >>> https://www.jiscmail.ac.uk/policyandsecurity/ >> >> >> To unsubscribe from the CCP4BB list, click the following link: >> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 >> >> This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing >> list hosted by www.jiscmail.ac.uk, terms & conditions are available at >> https://www.jiscmail.ac.uk/policyandsecurity/ > >-- >--- >Dr. Demetres D. Leonidas >Professor in Biochemistry >Department of Biochemistry & Biotechnology >University of Thessaly >Biopolis 41500 Larissa, Greece >- >Tel. +302410 565278 >Tel. +302410 565297 (Lab) >Tel. +304210 565263 (X-ray) >Fax. +302410 565290 >E-mail: ddleoni...@bio.uth.gr >ORCID ID orcid.org/-0002-3874-2523 >--- > > >-- >Αυτό το email έχει ελεγχθεί για ιούς από το Avast antivirus. >www.avast.com > > > >To unsubscribe from the CCP4BB list, click the following link: >https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > >This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing >list hosted by www.jiscmail.ac.uk, terms & conditions are available at >https://www.jiscmail.ac.uk/policyandsecurity/ To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/
Re: [ccp4bb] xds crashes
Dear Kay, you are right and I do apologize for the misunderstanding. Point 1 and 2 are sufficient. best Demetres On 11/29/2022 10:00 AM, Kay Diederichs wrote: Dear Demetres, sounds a bit drastic! The second point alone would certainly suffice, because then no forking is done. What kind of hardware is that? I would expect that on a contemporary workstation you have 8GB of RAM or more; that would allow for 8 or more processes. Do you have a swapfile? If not, create one. Please check out https://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/Performance Best wishes, Kay On Tue, 29 Nov 2022 09:34:49 +0200, Demetres D. Leonidas wrote: Dear Kay, after some digging and help from Max Nanao we found that the problem was the memory. It was fulling rapidly. The workaround was to: 1. unzip the cbf files prior to processing 2. use maximum number of jobs 1 3. use number of processors 4 4. run xds_par Many thanks for all your help Demetres On 11/29/2022 9:03 AM, Kay Diederichs wrote: Dear Demetres, I agree with what James says: this is the operating system trying to to prevent, in an over-zealous way, the forking of too many or too big (in terms of memory) processes. In other words, XDS hasn't changed, but the operating system changed - it applies different limits to its processes. Your friendly system administrator should try to find out exactly which limit(s) changed, and reset the value(s). Perhaps a hint is in the Release Notes of Ubuntu 22.04 ? Pls report here when you've found it - currently you are the only one reporting it. Best wishes, Kay On Mon, 28 Nov 2022 12:06:43 -0800, James Holton wrote: Sounds like your kernel might think "forkxds" is creating a "fork bomb". Too many sub-processes firing off in too short a time triggers this. On one of my systems, I fixed it by editing /etc/security/limits.d/20-nproc.conf so that users are allowed ~10x more pids than normal. This tends to prevent these mysterious "Killed" errors. HTH? -James Holton MAD Scientist On 11/28/2022 7:44 AM, Demetres D. Leonidas wrote: Dear all, I do not know if this is the right list and I would like to apologize if it is not. We have repeatedly experienced xds crashes in machines running ubuntu 22.04.1 with the following message repeated several times at the INTEGRATE step /usr/local/bin/forkxds: line 60: 3427 Done echo "$itask" 3428 Killed | $amain We are trying to process data from P13 at EMBL-Hamburg. We are running XDS version Jan 10, 2022 BUILT=20220820 Any ideas ? Demetres To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/ To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/ -- --- Dr. Demetres D. Leonidas Professor in Biochemistry Department of Biochemistry & Biotechnology University of Thessaly Biopolis 41500 Larissa, Greece - Tel. +302410 565278 Tel. +302410 565297 (Lab) Tel. +304210 565263 (X-ray) Fax. +302410 565290 E-mail: ddleoni...@bio.uth.gr ORCID ID orcid.org/-0002-3874-2523 --- -- Αυτό το email έχει ελεγχθεί για ιούς από το Avast antivirus. www.avast.com To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/ To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/ -- --- Dr. Demetres D. Leonidas Professor in Biochemistry Department of Biochemistry & Biotechnology University of Thessaly Biopolis 41500 Larissa, Greece - Tel. +302410 565278 Tel. +302410 565297 (Lab)
Re: [ccp4bb] xds crashes
Dear Demetres, On Tue, Nov 29, 2022 at 10:25:17AM +0200, Demetres D. Leonidas wrote: > Dear Kay, > > you are right and I do apologize for the misunderstanding. Point 1 > and 2 are sufficient. You can probably avoid doing any of this by using the LIB= feature of XDS on gzipped CBF files with the excellent xds-zcbf plugin [1]. If you have autoPROC available this will be used automatically [2]. Regards, ClAuS & Clemens [1] https://git.embl.de/nikolova/xds-zcbf [2] https://www.globalphasing.com/autoproc > > best > > Demetres > > On 11/29/2022 10:00 AM, Kay Diederichs wrote: > >Dear Demetres, > > > >sounds a bit drastic! > >The second point alone would certainly suffice, because then no forking is > >done. > > > >What kind of hardware is that? > >I would expect that on a contemporary workstation you have 8GB of RAM or > >more; that would allow for 8 or more processes. > >Do you have a swapfile? If not, create one. > > > >Please check out > >https://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/Performance > > > >Best wishes, > >Kay > > > > > >On Tue, 29 Nov 2022 09:34:49 +0200, Demetres D. Leonidas > > wrote: > > > >>Dear Kay, > >> > >>after some digging and help from Max Nanao we found that the problem > >>was the memory. It was fulling rapidly. The workaround was to: > >> > >>1. unzip the cbf files prior to processing > >> > >>2. use maximum number of jobs 1 > >> > >>3. use number of processors 4 > >> > >>4. run xds_par > >> > >>Many thanks for all your help > >> > >>Demetres > >> > >>On 11/29/2022 9:03 AM, Kay Diederichs wrote: > >>>Dear Demetres, > >>> > >>>I agree with what James says: this is the operating system trying to to > >>>prevent, in an over-zealous way, the > >>>forking of too many or too big (in terms of memory) processes. > >>>In other words, XDS hasn't changed, but the operating system changed - it > >>>applies different limits to its processes. > >>>Your friendly system administrator should try to find out exactly which > >>>limit(s) changed, and reset the value(s). > >>>Perhaps a hint is in the Release Notes of Ubuntu 22.04 ? > >>> > >>>Pls report here when you've found it - currently you are the only one > >>>reporting it. > >>> > >>>Best wishes, > >>>Kay > >>> > >>> > >>>On Mon, 28 Nov 2022 12:06:43 -0800, James Holton wrote: > >>> > Sounds like your kernel might think "forkxds" is creating a "fork > bomb". Too many sub-processes firing off in too short a time triggers > this. On one of my systems, I fixed it by editing > /etc/security/limits.d/20-nproc.conf so that users are allowed ~10x more > pids than normal. This tends to prevent these mysterious "Killed" errors. > > HTH? > > -James Holton > MAD Scientist > > > On 11/28/2022 7:44 AM, Demetres D. Leonidas wrote: > >Dear all, > > > >I do not know if this is the right list and I would like to apologize > >if it is not. > > > >We have repeatedly experienced xds crashes in machines running ubuntu > >22.04.1 with the following message repeated several times at the > >INTEGRATE step > > > >/usr/local/bin/forkxds: line 60: 3427 Done echo "$itask" > > > > 3428 Killed | $amain > > > >We are trying to process data from P13 at EMBL-Hamburg. > > > >We are running XDS version Jan 10, 2022 BUILT=20220820 > > > >Any ideas ? > > > >Demetres > > > > > To unsubscribe from the CCP4BB list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > > This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a > mailing list hosted by www.jiscmail.ac.uk, terms & conditions are > available at https://www.jiscmail.ac.uk/policyandsecurity/ > >>> > >>> > >>>To unsubscribe from the CCP4BB list, click the following link: > >>>https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > >>> > >>>This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing > >>>list hosted by www.jiscmail.ac.uk, terms & conditions are available at > >>>https://www.jiscmail.ac.uk/policyandsecurity/ > >>-- > >>--- > >>Dr. Demetres D. Leonidas > >>Professor in Biochemistry > >>Department of Biochemistry & Biotechnology > >>University of Thessaly > >>Biopolis 41500 Larissa, Greece > >>- > >>Tel. +302410 565278 > >>Tel. +302410 565297 (Lab) > >>Tel. +304210 565263 (X-ray) > >>Fax. +302410 565290 > >>E-mail: ddleoni...@bio.uth.gr > >>ORCID ID orcid.org/-0002-3874-2523 > >>--- > >> > >> > >>-- > >> email ?? ?? > >>Avast antivirus. > >>www.avast.com > >> > >
Re: [ccp4bb] xds crashes
Dear Demetres, I googled the Ubuntu 22.04 release notes and found https://discourse.ubuntu.com/t/jammy-jellyfish-release-notes/24668 which says "We’ve enabled the userspace OOMD service and are shipping the systemd-oomd package by default on the “Ubuntu Desktop” flavour, to avoid overloaded systems and the need of the kernel’s OOM killer to kick in. The OOMD status can be checked using oomctl." Then googling "Ubuntu 22.04 oomctl" takes me e.g. to https://askubuntu.com/questions/1405021/firefox-closes-by-itself-in-ubuntu-22-04 and https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1972159 which confirms that there may be a problem in Ubuntu 22.04 with processes being killed without good reason, and some hints on how to configure your system to avoid that. Maybe the systems-oomd package can simply be removed. Hope that helps, Kay On Tue, 29 Nov 2022 10:25:17 +0200, Demetres D. Leonidas wrote: >Dear Kay, > >you are right and I do apologize for the misunderstanding. Point 1 and 2 >are sufficient. > >best > >Demetres > >On 11/29/2022 10:00 AM, Kay Diederichs wrote: >> Dear Demetres, >> >> sounds a bit drastic! >> The second point alone would certainly suffice, because then no forking is >> done. >> >> What kind of hardware is that? >> I would expect that on a contemporary workstation you have 8GB of RAM or >> more; that would allow for 8 or more processes. >> Do you have a swapfile? If not, create one. >> >> Please check out >> https://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/Performance >> >> Best wishes, >> Kay >> >> >> On Tue, 29 Nov 2022 09:34:49 +0200, Demetres D. Leonidas >> wrote: >> >>> Dear Kay, >>> >>> after some digging and help from Max Nanao we found that the problem >>> was the memory. It was fulling rapidly. The workaround was to: >>> >>> 1. unzip the cbf files prior to processing >>> >>> 2. use maximum number of jobs 1 >>> >>> 3. use number of processors 4 >>> >>> 4. run xds_par >>> >>> Many thanks for all your help >>> >>> Demetres >>> >>> On 11/29/2022 9:03 AM, Kay Diederichs wrote: Dear Demetres, I agree with what James says: this is the operating system trying to to prevent, in an over-zealous way, the forking of too many or too big (in terms of memory) processes. In other words, XDS hasn't changed, but the operating system changed - it applies different limits to its processes. Your friendly system administrator should try to find out exactly which limit(s) changed, and reset the value(s). Perhaps a hint is in the Release Notes of Ubuntu 22.04 ? Pls report here when you've found it - currently you are the only one reporting it. Best wishes, Kay On Mon, 28 Nov 2022 12:06:43 -0800, James Holton wrote: > Sounds like your kernel might think "forkxds" is creating a "fork > bomb". Too many sub-processes firing off in too short a time triggers > this. On one of my systems, I fixed it by editing > /etc/security/limits.d/20-nproc.conf so that users are allowed ~10x more > pids than normal. This tends to prevent these mysterious "Killed" errors. > > HTH? > > -James Holton > MAD Scientist > > > On 11/28/2022 7:44 AM, Demetres D. Leonidas wrote: >> Dear all, >> >> I do not know if this is the right list and I would like to apologize >> if it is not. >> >> We have repeatedly experienced xds crashes in machines running ubuntu >> 22.04.1 with the following message repeated several times at the >> INTEGRATE step >> >> /usr/local/bin/forkxds: line 60: 3427 Done echo "$itask" >> >> 3428 Killed | $amain >> >> We are trying to process data from P13 at EMBL-Hamburg. >> >> We are running XDS version Jan 10, 2022 BUILT=20220820 >> >> Any ideas ? >> >> Demetres >> > > > To unsubscribe from the CCP4BB list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > > This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a > mailing list hosted by www.jiscmail.ac.uk, terms & conditions are > available at https://www.jiscmail.ac.uk/policyandsecurity/ To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/ >>> -- >>> --- >>> Dr. Demetres D. Leonidas >>> Professor in Biochemistry >>> Department of Biochemistry & Biotechnology >>> University of Thes
Re: [ccp4bb] xds crashes
Dear Kay, many thanks for your suggestions. We will look into this. best Demetres On 11/29/2022 2:35 PM, Kay Diederichs wrote: Dear Demetres, I googled the Ubuntu 22.04 release notes and found https://discourse.ubuntu.com/t/jammy-jellyfish-release-notes/24668 which says "We’ve enabled the userspace OOMD service and are shipping the systemd-oomd package by default on the “Ubuntu Desktop” flavour, to avoid overloaded systems and the need of the kernel’s OOM killer to kick in. The OOMD status can be checked using oomctl." Then googling "Ubuntu 22.04 oomctl" takes me e.g. to https://askubuntu.com/questions/1405021/firefox-closes-by-itself-in-ubuntu-22-04 and https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1972159 which confirms that there may be a problem in Ubuntu 22.04 with processes being killed without good reason, and some hints on how to configure your system to avoid that. Maybe the systems-oomd package can simply be removed. Hope that helps, Kay On Tue, 29 Nov 2022 10:25:17 +0200, Demetres D. Leonidas wrote: Dear Kay, you are right and I do apologize for the misunderstanding. Point 1 and 2 are sufficient. best Demetres On 11/29/2022 10:00 AM, Kay Diederichs wrote: Dear Demetres, sounds a bit drastic! The second point alone would certainly suffice, because then no forking is done. What kind of hardware is that? I would expect that on a contemporary workstation you have 8GB of RAM or more; that would allow for 8 or more processes. Do you have a swapfile? If not, create one. Please check out https://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/Performance Best wishes, Kay On Tue, 29 Nov 2022 09:34:49 +0200, Demetres D. Leonidas wrote: Dear Kay, after some digging and help from Max Nanao we found that the problem was the memory. It was fulling rapidly. The workaround was to: 1. unzip the cbf files prior to processing 2. use maximum number of jobs 1 3. use number of processors 4 4. run xds_par Many thanks for all your help Demetres On 11/29/2022 9:03 AM, Kay Diederichs wrote: Dear Demetres, I agree with what James says: this is the operating system trying to to prevent, in an over-zealous way, the forking of too many or too big (in terms of memory) processes. In other words, XDS hasn't changed, but the operating system changed - it applies different limits to its processes. Your friendly system administrator should try to find out exactly which limit(s) changed, and reset the value(s). Perhaps a hint is in the Release Notes of Ubuntu 22.04 ? Pls report here when you've found it - currently you are the only one reporting it. Best wishes, Kay On Mon, 28 Nov 2022 12:06:43 -0800, James Holton wrote: Sounds like your kernel might think "forkxds" is creating a "fork bomb". Too many sub-processes firing off in too short a time triggers this. On one of my systems, I fixed it by editing /etc/security/limits.d/20-nproc.conf so that users are allowed ~10x more pids than normal. This tends to prevent these mysterious "Killed" errors. HTH? -James Holton MAD Scientist On 11/28/2022 7:44 AM, Demetres D. Leonidas wrote: Dear all, I do not know if this is the right list and I would like to apologize if it is not. We have repeatedly experienced xds crashes in machines running ubuntu 22.04.1 with the following message repeated several times at the INTEGRATE step /usr/local/bin/forkxds: line 60: 3427 Done echo "$itask" 3428 Killed | $amain We are trying to process data from P13 at EMBL-Hamburg. We are running XDS version Jan 10, 2022 BUILT=20220820 Any ideas ? Demetres To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/ To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/ -- --- Dr. Demetres D. Leonidas Professor in Biochemistry Department of Biochemistry & Biotechnology University of Thessaly Biopolis 41500 Larissa, Greece - Tel. +302410 565278 Tel. +302410 565297 (Lab) Tel. +304210 565263 (X-ray) Fax. +302410 565290 E-mail: ddleoni...@bio.uth.gr ORCID ID orcid.org/-0002-3874-2523 --- -- Αυτό το email έχει ελεγχθεί για ιούς από το Avast antivirus. www.avast.com
