RE: New Policeman Jenkins Hardware: AMD Ryzen 7 3700X Octa-Core, DDR4 ECC, 2x1 Terabyte NVME SSD as RAID
Hi Robert, +1 One small thing: It should use java.io.tmpdir, but append its own process id or something like that to it, so multiple runs (like on Jeekins) in parallel do not override each other. Maybe just use tmpname (the corresponding java.nio.file.Files function) or similar to generate a new temporary directory that’s unique. I think ANT can do this for us (I think there is an ANT task to get a temporary name, which is assigned to a property). Uwe - Uwe Schindler Achterdiek 19, D-28357 Bremen https://www.thetaphi.de eMail: u...@thetaphi.de From: Robert Muir Sent: Sunday, January 19, 2020 11:47 PM To: dev@lucene.apache.org Subject: Re: New Policeman Jenkins Hardware: AMD Ryzen 7 3700X Octa-Core, DDR4 ECC, 2x1 Terabyte NVME SSD as RAID agreed. for all situations it may not help. but for my test 0 bytes were left at the end. i watched the io and usage as tests ran and the usage only even peaks at megabytes. just death by a thousand stabs... literally. gazillions of tests writing only a few MB each. i think we can all agree these test indexes are "temp data"? so if the build respects java.io.tmpdir here, then imo it behaves nicely and gives user the chance to prevent hardware destruction. On Sun, Jan 19, 2020, 3:36 PM Dawid Weiss mailto:dawid.we...@gmail.com> > wrote: Maybe we should open an issue. For example it would be good to fix the gradle build here too. It's definitely possible to relocate those folders (gradle-wise). I don't think it buys anything for Windows users or for those who have /tmp mounted on a physical drive but if you know what you're doing then it should also make tests faster. Dawid
Re: New Policeman Jenkins Hardware: AMD Ryzen 7 3700X Octa-Core, DDR4 ECC, 2x1 Terabyte NVME SSD as RAID
agreed. for all situations it may not help. but for my test 0 bytes were left at the end. i watched the io and usage as tests ran and the usage only even peaks at megabytes. just death by a thousand stabs... literally. gazillions of tests writing only a few MB each. i think we can all agree these test indexes are "temp data"? so if the build respects java.io.tmpdir here, then imo it behaves nicely and gives user the chance to prevent hardware destruction. On Sun, Jan 19, 2020, 3:36 PM Dawid Weiss wrote: > > >> Maybe we should open an issue. For example it would be good to fix the >> gradle build here too. >> > > It's definitely possible to relocate those folders (gradle-wise). I don't > think it buys anything for Windows users or for those who have /tmp mounted > on a physical drive but if you know what you're doing then it should also > make tests faster. > > Dawid >
Re: New Policeman Jenkins Hardware: AMD Ryzen 7 3700X Octa-Core, DDR4 ECC, 2x1 Terabyte NVME SSD as RAID
...and safe your SSD from death. Some statistics: in 12 hours the macOS slave wrote 114 Gigabytes to it's virtual harddisk - according to Darwin's top implementation, which displays disk IO since bootup of kernel. Uwe Am January 19, 2020 8:35:55 PM UTC schrieb Dawid Weiss : >> Maybe we should open an issue. For example it would be good to fix >the >> gradle build here too. >> > >It's definitely possible to relocate those folders (gradle-wise). I >don't >think it buys anything for Windows users or for those who have /tmp >mounted >on a physical drive but if you know what you're doing then it should >also >make tests faster. > >Dawid -- Uwe Schindler Achterdiek 19, 28357 Bremen https://www.thetaphi.de
Re: New Policeman Jenkins Hardware: AMD Ryzen 7 3700X Octa-Core, DDR4 ECC, 2x1 Terabyte NVME SSD as RAID
> Maybe we should open an issue. For example it would be good to fix the > gradle build here too. > It's definitely possible to relocate those folders (gradle-wise). I don't think it buys anything for Windows users or for those who have /tmp mounted on a physical drive but if you know what you're doing then it should also make tests faster. Dawid
Re: New Policeman Jenkins Hardware: AMD Ryzen 7 3700X Octa-Core, DDR4 ECC, 2x1 Terabyte NVME SSD as RAID
On Thu, Jan 16, 2020 at 2:38 AM Uwe Schindler wrote: > So Lucene and Solr tests are eating your SSD, be aware of that!!! > > By default all the tests are writing indexes to build/ directory where your source checkout is, I think this is a bad default? As a workaround I can manually pass -Dtests.workDir=/tmp/folder to 'ant' so that test temporary data only hits a tmpfs mount. All the other stuff (build classes, test xmls, etc) still goes to the usual places, just not the gigabytes of indexes being created and destroyed. Maybe we should open an issue. For example it would be good to fix the gradle build here too.
RE: New Policeman Jenkins Hardware: AMD Ryzen 7 3700X Octa-Core, DDR4 ECC, 2x1 Terabyte NVME SSD as RAID
Sorry false positive, Solaris hung again. Uwe - Uwe Schindler Achterdiek 19, D-28357 Bremen <https://www.thetaphi.de> https://www.thetaphi.de eMail: u...@thetaphi.de From: Uwe Schindler Sent: Friday, January 17, 2020 7:21 PM To: dev@lucene.apache.org Subject: RE: New Policeman Jenkins Hardware: AMD Ryzen 7 3700X Octa-Core, DDR4 ECC, 2x1 Terabyte NVME SSD as RAID Hi, I updated the virtualization on the server. It looks like Solaris is working again. But I have seen only one build yet – which passed without hangs. Builds are back enabled. Uwe - Uwe Schindler Achterdiek 19, D-28357 Bremen <https://www.thetaphi.de> https://www.thetaphi.de eMail: u...@thetaphi.de <mailto:u...@thetaphi.de> From: Uwe Schindler mailto:u...@thetaphi.de> > Sent: Thursday, January 16, 2020 9:30 AM To: dev@lucene.apache.org <mailto:dev@lucene.apache.org> Subject: RE: New Policeman Jenkins Hardware: AMD Ryzen 7 3700X Octa-Core, DDR4 ECC, 2x1 Terabyte NVME SSD as RAID Actually the Solaris builds are sometimes hanging forever (more often with new hardware). So they don’t pollute mailinglist they just block executor. I have to manually kill them. Uwe - Uwe Schindler Achterdiek 19, D-28357 Bremen <https://www.thetaphi.de> https://www.thetaphi.de eMail: u...@thetaphi.de <mailto:u...@thetaphi.de> From: Dawid Weiss mailto:dawid.we...@gmail.com> > Sent: Thursday, January 16, 2020 9:24 AM To: Lucene Dev mailto:dev@lucene.apache.org> > Subject: Re: New Policeman Jenkins Hardware: AMD Ryzen 7 3700X Octa-Core, DDR4 ECC, 2x1 Terabyte NVME SSD as RAID With new CPU, builds seem up to 2 times faster. FYI, the Linux builds are running with tests.multiplicator=3 to better trigger JVM failures. You have to remind about that when you reproduce failures - it looks like this is not printed in reproduce lines. It is printed with gradle builds (any non-defaults are). The option is "tests.multiplier" actually; check if you don't have a typo there? But we can also drop Solaris builds, as they only work with 8.x - no Java 11 support on Solaris anymore by Oracle. Any comments? As somebody who tries to look at each failure sent to the mailing list (at least recently...) I still think the EA builds that are KNOWN to be buggy should be disabled until next upgrade. They pollute the mailing list with so many identical errors that this discourages one from looking through the logs. Dawid
RE: New Policeman Jenkins Hardware: AMD Ryzen 7 3700X Octa-Core, DDR4 ECC, 2x1 Terabyte NVME SSD as RAID
Hi, I updated the virtualization on the server. It looks like Solaris is working again. But I have seen only one build yet – which passed without hangs. Builds are back enabled. Uwe - Uwe Schindler Achterdiek 19, D-28357 Bremen <https://www.thetaphi.de> https://www.thetaphi.de eMail: u...@thetaphi.de From: Uwe Schindler Sent: Thursday, January 16, 2020 9:30 AM To: dev@lucene.apache.org Subject: RE: New Policeman Jenkins Hardware: AMD Ryzen 7 3700X Octa-Core, DDR4 ECC, 2x1 Terabyte NVME SSD as RAID Actually the Solaris builds are sometimes hanging forever (more often with new hardware). So they don’t pollute mailinglist they just block executor. I have to manually kill them. Uwe - Uwe Schindler Achterdiek 19, D-28357 Bremen <https://www.thetaphi.de> https://www.thetaphi.de eMail: u...@thetaphi.de <mailto:u...@thetaphi.de> From: Dawid Weiss mailto:dawid.we...@gmail.com> > Sent: Thursday, January 16, 2020 9:24 AM To: Lucene Dev mailto:dev@lucene.apache.org> > Subject: Re: New Policeman Jenkins Hardware: AMD Ryzen 7 3700X Octa-Core, DDR4 ECC, 2x1 Terabyte NVME SSD as RAID With new CPU, builds seem up to 2 times faster. FYI, the Linux builds are running with tests.multiplicator=3 to better trigger JVM failures. You have to remind about that when you reproduce failures - it looks like this is not printed in reproduce lines. It is printed with gradle builds (any non-defaults are). The option is "tests.multiplier" actually; check if you don't have a typo there? But we can also drop Solaris builds, as they only work with 8.x - no Java 11 support on Solaris anymore by Oracle. Any comments? As somebody who tries to look at each failure sent to the mailing list (at least recently...) I still think the EA builds that are KNOWN to be buggy should be disabled until next upgrade. They pollute the mailing list with so many identical errors that this discourages one from looking through the logs. Dawid
Re: New Policeman Jenkins Hardware: AMD Ryzen 7 3700X Octa-Core, DDR4 ECC, 2x1 Terabyte NVME SSD as RAID
>> It is printed with gradle builds (any non-defaults are). The option is >> "tests.multiplier" actually; check if you don't have a typo there? I think something like this could be added with relative ease. I don't know how much useful it is though. :) D. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: New Policeman Jenkins Hardware: AMD Ryzen 7 3700X Octa-Core, DDR4 ECC, 2x1 Terabyte NVME SSD as RAID
On Thu, Jan 16, 2020 at 3:25 AM Dawid Weiss wrote: > > >> With new CPU, builds seem up to 2 times faster. FYI, the Linux builds are >> running with tests.multiplicator=3 to better trigger JVM failures. You have >> to remind about that when you reproduce failures - it looks like this is >> not printed in reproduce lines. >> > > It is printed with gradle builds (any non-defaults are). The option is > "tests.multiplier" actually; check if you don't have a typo there? > Hmm if we do accidentally have a typo, will the new gradle build catch it? Or is it still lenient, like the ant build? > But we can also drop Solaris builds, as they only work with 8.x - no Java >> 11 support on Solaris anymore by Oracle. Any comments? > > > As somebody who tries to look at each failure sent to the mailing list (at > least recently...) I still think the EA builds that are KNOWN to be buggy > should be disabled until next upgrade. They pollute the mailing list with > so many identical errors that this discourages one from looking through the > logs. > +1 Mike McCandless http://blog.mikemccandless.com >
RE: New Policeman Jenkins Hardware: AMD Ryzen 7 3700X Octa-Core, DDR4 ECC, 2x1 Terabyte NVME SSD as RAID
Actually it’s printed in reproduce line. Sorry for the false alarm. - Uwe Schindler Achterdiek 19, D-28357 Bremen <https://www.thetaphi.de> https://www.thetaphi.de eMail: u...@thetaphi.de From: Uwe Schindler Sent: Thursday, January 16, 2020 9:29 AM To: dev@lucene.apache.org Subject: RE: New Policeman Jenkins Hardware: AMD Ryzen 7 3700X Octa-Core, DDR4 ECC, 2x1 Terabyte NVME SSD as RAID $ cat lucene.build.properties tests.jvms=6 tests.multiplier=3 - UWE SCHINDLER Software Architecture, Apache Lucene, Elasticsearch PANGAEA - Data Publisher for Earth & Environmental Science MARUM (FVG Ost building) - University of Bremen Room 0210, Leobener Str. 2, D-28359 Bremen Tel.: +49 421 218 65595 Fax: +49 421 218 65505 <https://www.pangaea.de/> https://www.pangaea.de/ E-mail: uschind...@pangaea.de <mailto:uschind...@pangaea.de> From: Dawid Weiss mailto:dawid.we...@gmail.com> > Sent: Thursday, January 16, 2020 9:24 AM To: Lucene Dev mailto:dev@lucene.apache.org> > Subject: Re: New Policeman Jenkins Hardware: AMD Ryzen 7 3700X Octa-Core, DDR4 ECC, 2x1 Terabyte NVME SSD as RAID With new CPU, builds seem up to 2 times faster. FYI, the Linux builds are running with tests.multiplicator=3 to better trigger JVM failures. You have to remind about that when you reproduce failures - it looks like this is not printed in reproduce lines. It is printed with gradle builds (any non-defaults are). The option is "tests.multiplier" actually; check if you don't have a typo there? But we can also drop Solaris builds, as they only work with 8.x - no Java 11 support on Solaris anymore by Oracle. Any comments? As somebody who tries to look at each failure sent to the mailing list (at least recently...) I still think the EA builds that are KNOWN to be buggy should be disabled until next upgrade. They pollute the mailing list with so many identical errors that this discourages one from looking through the logs. Dawid
RE: New Policeman Jenkins Hardware: AMD Ryzen 7 3700X Octa-Core, DDR4 ECC, 2x1 Terabyte NVME SSD as RAID
Actually the Solaris builds are sometimes hanging forever (more often with new hardware). So they don’t pollute mailinglist they just block executor. I have to manually kill them. Uwe - Uwe Schindler Achterdiek 19, D-28357 Bremen <https://www.thetaphi.de> https://www.thetaphi.de eMail: u...@thetaphi.de From: Dawid Weiss Sent: Thursday, January 16, 2020 9:24 AM To: Lucene Dev Subject: Re: New Policeman Jenkins Hardware: AMD Ryzen 7 3700X Octa-Core, DDR4 ECC, 2x1 Terabyte NVME SSD as RAID With new CPU, builds seem up to 2 times faster. FYI, the Linux builds are running with tests.multiplicator=3 to better trigger JVM failures. You have to remind about that when you reproduce failures - it looks like this is not printed in reproduce lines. It is printed with gradle builds (any non-defaults are). The option is "tests.multiplier" actually; check if you don't have a typo there? But we can also drop Solaris builds, as they only work with 8.x - no Java 11 support on Solaris anymore by Oracle. Any comments? As somebody who tries to look at each failure sent to the mailing list (at least recently...) I still think the EA builds that are KNOWN to be buggy should be disabled until next upgrade. They pollute the mailing list with so many identical errors that this discourages one from looking through the logs. Dawid
RE: New Policeman Jenkins Hardware: AMD Ryzen 7 3700X Octa-Core, DDR4 ECC, 2x1 Terabyte NVME SSD as RAID
$ cat lucene.build.properties tests.jvms=6 tests.multiplier=3 - UWE SCHINDLER Software Architecture, Apache Lucene, Elasticsearch PANGAEA - Data Publisher for Earth & Environmental Science MARUM (FVG Ost building) - University of Bremen Room 0210, Leobener Str. 2, D-28359 Bremen Tel.: +49 421 218 65595 Fax: +49 421 218 65505 <https://www.pangaea.de/> https://www.pangaea.de/ E-mail: uschind...@pangaea.de From: Dawid Weiss Sent: Thursday, January 16, 2020 9:24 AM To: Lucene Dev Subject: Re: New Policeman Jenkins Hardware: AMD Ryzen 7 3700X Octa-Core, DDR4 ECC, 2x1 Terabyte NVME SSD as RAID With new CPU, builds seem up to 2 times faster. FYI, the Linux builds are running with tests.multiplicator=3 to better trigger JVM failures. You have to remind about that when you reproduce failures - it looks like this is not printed in reproduce lines. It is printed with gradle builds (any non-defaults are). The option is "tests.multiplier" actually; check if you don't have a typo there? But we can also drop Solaris builds, as they only work with 8.x - no Java 11 support on Solaris anymore by Oracle. Any comments? As somebody who tries to look at each failure sent to the mailing list (at least recently...) I still think the EA builds that are KNOWN to be buggy should be disabled until next upgrade. They pollute the mailing list with so many identical errors that this discourages one from looking through the logs. Dawid
Re: New Policeman Jenkins Hardware: AMD Ryzen 7 3700X Octa-Core, DDR4 ECC, 2x1 Terabyte NVME SSD as RAID
> With new CPU, builds seem up to 2 times faster. FYI, the Linux builds are > running with tests.multiplicator=3 to better trigger JVM failures. You have > to remind about that when you reproduce failures - it looks like this is > not printed in reproduce lines. > It is printed with gradle builds (any non-defaults are). The option is "tests.multiplier" actually; check if you don't have a typo there? > But we can also drop Solaris builds, as they only work with 8.x - no Java > 11 support on Solaris anymore by Oracle. Any comments? As somebody who tries to look at each failure sent to the mailing list (at least recently...) I still think the EA builds that are KNOWN to be buggy should be disabled until next upgrade. They pollute the mailing list with so many identical errors that this discourages one from looking through the logs. Dawid