[hibernate-dev] Analyzing the ORM testsuite
Hi all, I'm trying to analyze the memory usage of current master of Hibernate ORM; I started looking because of OOM errors on the PostgreSQL testsuite (as mentioned in another thread). So far I've seen that Mockito is allocating a "lot of stuff"; I might be able to improve some things in that area but it's a distraction from my goal as clearly we use Mockito even when not running the PostgreSQL backed tests and that seems to run fine. But it's polluting the data reports so making it hard (and slow!) to verify if there is a real issue. I could use some help to verify some tangential issues though; when running the testsuite locally it's taking much longer than what I've been used to see for the same testsuite some weeks ago, and I have some failures - regularly seeing the same failures. These are triggered by running a default build, on H2: > Task :hibernate-core:test org.hibernate.test.criteria.CriteriaLockingTest > testSetLockModeDifferentFromNONELogAWarnMessageWhenTheDialectUseFollowOnLockingIsTrue FAILED org.junit.runners.model.TestTimedOutException org.hibernate.test.criteria.CriteriaLockingTest > testSetLockModeNONEDoNotLogAWarnMessageWhenTheDialectUseFollowOnLockingIsTrue FAILED java.lang.Exception org.hibernate.test.criteria.CriteriaLockingTest > org.hibernate.test.criteria.CriteriaLockingTest FAILED java.lang.Exception org.hibernate.test.locking.warning.LockNoneWarmingTest > org.hibernate.test.locking.warning.LockNoneWarmingTest FAILED java.lang.Exception 6307 tests completed, 4 failed, 327 skipped Yet the puzzling point: on ci.hibernate.org I don't see these failures happening, nor the testsuite time seems to have increased of any significant amount (it regularly completes in somewhere between 15 and 20 minutes). It takes me more than an hour to get this result on a machine which would normally complete the testsuite in 8 minutes, and I got the same 4 failures when re-running it a second time. Could other developers also please checkout master - specifically at commit 291d4a3eeaa2ade32a42cfbcad5868b2114c34fe - and let me know if it works allright for you and if the build time is consistent with the usual times for your machine? I need to understand what might be different between my workstation and the CI server. I might even have some great improvements ready to reduce the testsuite time, but I'm not comfortable in committing them in this context. Thanks, Sanne ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Analyzing the ORM testsuite
Nothing off the top of my head. The build time is in normal range on my machine as of this morning (8 - 10 minutes). You mentioned a specific commit. Is that just when you noticed a change, or do you suspect something in that commit? I'm not at computer right now, but when I get back I will check out that commit and see if anything is strange locally. On Thu, Apr 19, 2018, 3:11 PM Sanne Grinovero wrote: > Hi all, > > I'm trying to analyze the memory usage of current master of Hibernate > ORM; I started looking because of OOM errors on the PostgreSQL > testsuite (as mentioned in another thread). > > So far I've seen that Mockito is allocating a "lot of stuff"; I might > be able to improve some things in that area but it's a distraction > from my goal as clearly we use Mockito even when not running the > PostgreSQL backed tests and that seems to run fine. But it's polluting > the data reports so making it hard (and slow!) to verify if there is a > real issue. > > I could use some help to verify some tangential issues though; when > running the testsuite locally it's taking much longer than what I've > been used to see for the same testsuite some weeks ago, and I have > some failures - regularly seeing the same failures. > > These are triggered by running a default build, on H2: > > > Task :hibernate-core:test > > org.hibernate.test.criteria.CriteriaLockingTest > > > testSetLockModeDifferentFromNONELogAWarnMessageWhenTheDialectUseFollowOnLockingIsTrue > FAILED > org.junit.runners.model.TestTimedOutException > > org.hibernate.test.criteria.CriteriaLockingTest > > > testSetLockModeNONEDoNotLogAWarnMessageWhenTheDialectUseFollowOnLockingIsTrue > FAILED > java.lang.Exception > > org.hibernate.test.criteria.CriteriaLockingTest > > org.hibernate.test.criteria.CriteriaLockingTest FAILED > java.lang.Exception > > org.hibernate.test.locking.warning.LockNoneWarmingTest > > org.hibernate.test.locking.warning.LockNoneWarmingTest FAILED > java.lang.Exception > > 6307 tests completed, 4 failed, 327 skipped > > Yet the puzzling point: on ci.hibernate.org I don't see these failures > happening, nor the testsuite time seems to have increased of any > significant amount (it regularly completes in somewhere between 15 and > 20 minutes). It takes me more than an hour to get this result on a > machine which would normally complete the testsuite in 8 minutes, and > I got the same 4 failures when re-running it a second time. > > Could other developers also please checkout master - specifically at > commit 291d4a3eeaa2ade32a42cfbcad5868b2114c34fe - and let me know if > it works allright for you and if the build time is consistent with the > usual times for your machine? I need to understand what might be > different between my workstation and the CI server. > > I might even have some great improvements ready to reduce the > testsuite time, but I'm not comfortable in committing them in this > context. > > Thanks, > Sanne > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev > ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Analyzing the ORM testsuite
Sanne - Are you running the build task or are you executing other more specific tasks and noticing the slowness? On 04/19/2018 04:25 PM, Steve Ebersole wrote: > Nothing off the top of my head. The build time is in normal range on my > machine as of this morning (8 - 10 minutes). > > You mentioned a specific commit. Is that just when you noticed a change, > or do you suspect something in that commit? > > I'm not at computer right now, but when I get back I will check out that > commit and see if anything is strange locally. > > > > On Thu, Apr 19, 2018, 3:11 PM Sanne Grinovero wrote: > >> Hi all, >> >> I'm trying to analyze the memory usage of current master of Hibernate >> ORM; I started looking because of OOM errors on the PostgreSQL >> testsuite (as mentioned in another thread). >> >> So far I've seen that Mockito is allocating a "lot of stuff"; I might >> be able to improve some things in that area but it's a distraction >> from my goal as clearly we use Mockito even when not running the >> PostgreSQL backed tests and that seems to run fine. But it's polluting >> the data reports so making it hard (and slow!) to verify if there is a >> real issue. >> >> I could use some help to verify some tangential issues though; when >> running the testsuite locally it's taking much longer than what I've >> been used to see for the same testsuite some weeks ago, and I have >> some failures - regularly seeing the same failures. >> >> These are triggered by running a default build, on H2: >> >>> Task :hibernate-core:test >> org.hibernate.test.criteria.CriteriaLockingTest > >> >> testSetLockModeDifferentFromNONELogAWarnMessageWhenTheDialectUseFollowOnLockingIsTrue >> FAILED >> org.junit.runners.model.TestTimedOutException >> >> org.hibernate.test.criteria.CriteriaLockingTest > >> >> testSetLockModeNONEDoNotLogAWarnMessageWhenTheDialectUseFollowOnLockingIsTrue >> FAILED >> java.lang.Exception >> >> org.hibernate.test.criteria.CriteriaLockingTest > >> org.hibernate.test.criteria.CriteriaLockingTest FAILED >> java.lang.Exception >> >> org.hibernate.test.locking.warning.LockNoneWarmingTest > >> org.hibernate.test.locking.warning.LockNoneWarmingTest FAILED >> java.lang.Exception >> >> 6307 tests completed, 4 failed, 327 skipped >> >> Yet the puzzling point: on ci.hibernate.org I don't see these failures >> happening, nor the testsuite time seems to have increased of any >> significant amount (it regularly completes in somewhere between 15 and >> 20 minutes). It takes me more than an hour to get this result on a >> machine which would normally complete the testsuite in 8 minutes, and >> I got the same 4 failures when re-running it a second time. >> >> Could other developers also please checkout master - specifically at >> commit 291d4a3eeaa2ade32a42cfbcad5868b2114c34fe - and let me know if >> it works allright for you and if the build time is consistent with the >> usual times for your machine? I need to understand what might be >> different between my workstation and the CI server. >> >> I might even have some great improvements ready to reduce the >> testsuite time, but I'm not comfortable in committing them in this >> context. >> >> Thanks, >> Sanne >> ___ >> hibernate-dev mailing list >> hibernate-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Analyzing the ORM testsuite
On 19 April 2018 at 21:25, Steve Ebersole wrote: > Nothing off the top of my head. The build time is in normal range on my > machine as of this morning (8 - 10 minutes). Great > You mentioned a specific commit. Is that just when you noticed a change, or > do you suspect something in that commit? Not suspecting anything yet, I just mentioned it so that people could try reproduce in the most similar context. > I'm not at computer right now, but when I get back I will check out that > commit and see if anything is strange locally. Don't worry on the specific commit, it's very useful to know it works "in normal ranges" for others. Thanks, Sanne > > > > On Thu, Apr 19, 2018, 3:11 PM Sanne Grinovero wrote: >> >> Hi all, >> >> I'm trying to analyze the memory usage of current master of Hibernate >> ORM; I started looking because of OOM errors on the PostgreSQL >> testsuite (as mentioned in another thread). >> >> So far I've seen that Mockito is allocating a "lot of stuff"; I might >> be able to improve some things in that area but it's a distraction >> from my goal as clearly we use Mockito even when not running the >> PostgreSQL backed tests and that seems to run fine. But it's polluting >> the data reports so making it hard (and slow!) to verify if there is a >> real issue. >> >> I could use some help to verify some tangential issues though; when >> running the testsuite locally it's taking much longer than what I've >> been used to see for the same testsuite some weeks ago, and I have >> some failures - regularly seeing the same failures. >> >> These are triggered by running a default build, on H2: >> >> > Task :hibernate-core:test >> >> org.hibernate.test.criteria.CriteriaLockingTest > >> >> testSetLockModeDifferentFromNONELogAWarnMessageWhenTheDialectUseFollowOnLockingIsTrue >> FAILED >> org.junit.runners.model.TestTimedOutException >> >> org.hibernate.test.criteria.CriteriaLockingTest > >> >> testSetLockModeNONEDoNotLogAWarnMessageWhenTheDialectUseFollowOnLockingIsTrue >> FAILED >> java.lang.Exception >> >> org.hibernate.test.criteria.CriteriaLockingTest > >> org.hibernate.test.criteria.CriteriaLockingTest FAILED >> java.lang.Exception >> >> org.hibernate.test.locking.warning.LockNoneWarmingTest > >> org.hibernate.test.locking.warning.LockNoneWarmingTest FAILED >> java.lang.Exception >> >> 6307 tests completed, 4 failed, 327 skipped >> >> Yet the puzzling point: on ci.hibernate.org I don't see these failures >> happening, nor the testsuite time seems to have increased of any >> significant amount (it regularly completes in somewhere between 15 and >> 20 minutes). It takes me more than an hour to get this result on a >> machine which would normally complete the testsuite in 8 minutes, and >> I got the same 4 failures when re-running it a second time. >> >> Could other developers also please checkout master - specifically at >> commit 291d4a3eeaa2ade32a42cfbcad5868b2114c34fe - and let me know if >> it works allright for you and if the build time is consistent with the >> usual times for your machine? I need to understand what might be >> different between my workstation and the CI server. >> >> I might even have some great improvements ready to reduce the >> testsuite time, but I'm not comfortable in committing them in this >> context. >> >> Thanks, >> Sanne >> ___ >> hibernate-dev mailing list >> hibernate-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hibernate-dev ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Analyzing the ORM testsuite
On 19 April 2018 at 21:47, Chris Cranford wrote: > Sanne - > > Are you running the build task or are you executing other more specific > tasks and noticing the slowness? Just running "gradle clean build" from the root. I had the same results with OpenJDK 8 and 9, twice. Getting exactly the same failures as before. Thanks! > > On 04/19/2018 04:25 PM, Steve Ebersole wrote: > > Nothing off the top of my head. The build time is in normal range on my > machine as of this morning (8 - 10 minutes). > > You mentioned a specific commit. Is that just when you noticed a change, > or do you suspect something in that commit? > > I'm not at computer right now, but when I get back I will check out that > commit and see if anything is strange locally. > > > > On Thu, Apr 19, 2018, 3:11 PM Sanne Grinovero wrote: > > Hi all, > > I'm trying to analyze the memory usage of current master of Hibernate > ORM; I started looking because of OOM errors on the PostgreSQL > testsuite (as mentioned in another thread). > > So far I've seen that Mockito is allocating a "lot of stuff"; I might > be able to improve some things in that area but it's a distraction > from my goal as clearly we use Mockito even when not running the > PostgreSQL backed tests and that seems to run fine. But it's polluting > the data reports so making it hard (and slow!) to verify if there is a > real issue. > > I could use some help to verify some tangential issues though; when > running the testsuite locally it's taking much longer than what I've > been used to see for the same testsuite some weeks ago, and I have > some failures - regularly seeing the same failures. > > These are triggered by running a default build, on H2: > > Task :hibernate-core:test > > org.hibernate.test.criteria.CriteriaLockingTest > > > testSetLockModeDifferentFromNONELogAWarnMessageWhenTheDialectUseFollowOnLockingIsTrue > FAILED > org.junit.runners.model.TestTimedOutException > > org.hibernate.test.criteria.CriteriaLockingTest > > > testSetLockModeNONEDoNotLogAWarnMessageWhenTheDialectUseFollowOnLockingIsTrue > FAILED > java.lang.Exception > > org.hibernate.test.criteria.CriteriaLockingTest > > org.hibernate.test.criteria.CriteriaLockingTest FAILED > java.lang.Exception > > org.hibernate.test.locking.warning.LockNoneWarmingTest > > org.hibernate.test.locking.warning.LockNoneWarmingTest FAILED > java.lang.Exception > > 6307 tests completed, 4 failed, 327 skipped > > Yet the puzzling point: on ci.hibernate.org I don't see these failures > happening, nor the testsuite time seems to have increased of any > significant amount (it regularly completes in somewhere between 15 and > 20 minutes). It takes me more than an hour to get this result on a > machine which would normally complete the testsuite in 8 minutes, and > I got the same 4 failures when re-running it a second time. > > Could other developers also please checkout master - specifically at > commit 291d4a3eeaa2ade32a42cfbcad5868b2114c34fe - and let me know if > it works allright for you and if the build time is consistent with the > usual times for your machine? I need to understand what might be > different between my workstation and the CI server. > > I might even have some great improvements ready to reduce the > testsuite time, but I'm not comfortable in committing them in this > context. > > Thanks, > Sanne > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev > > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev > > ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Analyzing the ORM testsuite
The entire build process under OpenJDK 8 here: BUILD SUCCESSFUL in 47m 26s 251 actionable tasks: 233 executed, 18 up-to-date The only thing I noticed was that while executing the hibernate-core tests, the memory consumption seemed quite high toward the end of the test suite, particularly ~3.5GB memory consumed just to run the hibernate-core tests alone and that FindBugs complained about some magic numbers on some zip files while inspecting hibernate-osgi. Any idea if any substantial amount of that memory consumption related to the test framework tracking the output/errors of those executed tests? On 04/19/2018 05:20 PM, Sanne Grinovero wrote: > On 19 April 2018 at 21:47, Chris Cranford wrote: >> Sanne - >> >> Are you running the build task or are you executing other more specific >> tasks and noticing the slowness? > Just running "gradle clean build" from the root. I had the same > results with OpenJDK 8 and 9, twice. Getting exactly the same failures > as before. > > Thanks! > >> On 04/19/2018 04:25 PM, Steve Ebersole wrote: >> >> Nothing off the top of my head. The build time is in normal range on my >> machine as of this morning (8 - 10 minutes). >> >> You mentioned a specific commit. Is that just when you noticed a change, >> or do you suspect something in that commit? >> >> I'm not at computer right now, but when I get back I will check out that >> commit and see if anything is strange locally. >> >> >> >> On Thu, Apr 19, 2018, 3:11 PM Sanne Grinovero wrote: >> >> Hi all, >> >> I'm trying to analyze the memory usage of current master of Hibernate >> ORM; I started looking because of OOM errors on the PostgreSQL >> testsuite (as mentioned in another thread). >> >> So far I've seen that Mockito is allocating a "lot of stuff"; I might >> be able to improve some things in that area but it's a distraction >> from my goal as clearly we use Mockito even when not running the >> PostgreSQL backed tests and that seems to run fine. But it's polluting >> the data reports so making it hard (and slow!) to verify if there is a >> real issue. >> >> I could use some help to verify some tangential issues though; when >> running the testsuite locally it's taking much longer than what I've >> been used to see for the same testsuite some weeks ago, and I have >> some failures - regularly seeing the same failures. >> >> These are triggered by running a default build, on H2: >> >> Task :hibernate-core:test >> >> org.hibernate.test.criteria.CriteriaLockingTest > >> >> testSetLockModeDifferentFromNONELogAWarnMessageWhenTheDialectUseFollowOnLockingIsTrue >> FAILED >> org.junit.runners.model.TestTimedOutException >> >> org.hibernate.test.criteria.CriteriaLockingTest > >> >> testSetLockModeNONEDoNotLogAWarnMessageWhenTheDialectUseFollowOnLockingIsTrue >> FAILED >> java.lang.Exception >> >> org.hibernate.test.criteria.CriteriaLockingTest > >> org.hibernate.test.criteria.CriteriaLockingTest FAILED >> java.lang.Exception >> >> org.hibernate.test.locking.warning.LockNoneWarmingTest > >> org.hibernate.test.locking.warning.LockNoneWarmingTest FAILED >> java.lang.Exception >> >> 6307 tests completed, 4 failed, 327 skipped >> >> Yet the puzzling point: on ci.hibernate.org I don't see these failures >> happening, nor the testsuite time seems to have increased of any >> significant amount (it regularly completes in somewhere between 15 and >> 20 minutes). It takes me more than an hour to get this result on a >> machine which would normally complete the testsuite in 8 minutes, and >> I got the same 4 failures when re-running it a second time. >> >> Could other developers also please checkout master - specifically at >> commit 291d4a3eeaa2ade32a42cfbcad5868b2114c34fe - and let me know if >> it works allright for you and if the build time is consistent with the >> usual times for your machine? I need to understand what might be >> different between my workstation and the CI server. >> >> I might even have some great improvements ready to reduce the >> testsuite time, but I'm not comfortable in committing them in this >> context. >> >> Thanks, >> Sanne >> ___ >> hibernate-dev mailing list >> hibernate-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> >> ___ >> hibernate-dev mailing list >> hibernate-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> >> > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Analyzing the ORM testsuite
I'm not seeing anything unusual. I usually run `./gradlew clean test`. On Thu, Apr 19, 2018 at 2:20 PM, Sanne Grinovero wrote: > On 19 April 2018 at 21:47, Chris Cranford wrote: > > Sanne - > > > > Are you running the build task or are you executing other more specific > > tasks and noticing the slowness? > > Just running "gradle clean build" from the root. I had the same > results with OpenJDK 8 and 9, twice. Getting exactly the same failures > as before. > > Thanks! > > > > > On 04/19/2018 04:25 PM, Steve Ebersole wrote: > > > > Nothing off the top of my head. The build time is in normal range on my > > machine as of this morning (8 - 10 minutes). > > > > You mentioned a specific commit. Is that just when you noticed a change, > > or do you suspect something in that commit? > > > > I'm not at computer right now, but when I get back I will check out that > > commit and see if anything is strange locally. > > > > > > > > On Thu, Apr 19, 2018, 3:11 PM Sanne Grinovero > wrote: > > > > Hi all, > > > > I'm trying to analyze the memory usage of current master of Hibernate > > ORM; I started looking because of OOM errors on the PostgreSQL > > testsuite (as mentioned in another thread). > > > > So far I've seen that Mockito is allocating a "lot of stuff"; I might > > be able to improve some things in that area but it's a distraction > > from my goal as clearly we use Mockito even when not running the > > PostgreSQL backed tests and that seems to run fine. But it's polluting > > the data reports so making it hard (and slow!) to verify if there is a > > real issue. > > > > I could use some help to verify some tangential issues though; when > > running the testsuite locally it's taking much longer than what I've > > been used to see for the same testsuite some weeks ago, and I have > > some failures - regularly seeing the same failures. > > > > These are triggered by running a default build, on H2: > > > > Task :hibernate-core:test > > > > org.hibernate.test.criteria.CriteriaLockingTest > > > > > testSetLockModeDifferentFromNONELogAWarnMessageWhenTheDialec > tUseFollowOnLockingIsTrue > > FAILED > > org.junit.runners.model.TestTimedOutException > > > > org.hibernate.test.criteria.CriteriaLockingTest > > > > > testSetLockModeNONEDoNotLogAWarnMessageWhenTheDialectUseFoll > owOnLockingIsTrue > > FAILED > > java.lang.Exception > > > > org.hibernate.test.criteria.CriteriaLockingTest > > > org.hibernate.test.criteria.CriteriaLockingTest FAILED > > java.lang.Exception > > > > org.hibernate.test.locking.warning.LockNoneWarmingTest > > > org.hibernate.test.locking.warning.LockNoneWarmingTest FAILED > > java.lang.Exception > > > > 6307 tests completed, 4 failed, 327 skipped > > > > Yet the puzzling point: on ci.hibernate.org I don't see these failures > > happening, nor the testsuite time seems to have increased of any > > significant amount (it regularly completes in somewhere between 15 and > > 20 minutes). It takes me more than an hour to get this result on a > > machine which would normally complete the testsuite in 8 minutes, and > > I got the same 4 failures when re-running it a second time. > > > > Could other developers also please checkout master - specifically at > > commit 291d4a3eeaa2ade32a42cfbcad5868b2114c34fe - and let me know if > > it works allright for you and if the build time is consistent with the > > usual times for your machine? I need to understand what might be > > different between my workstation and the CI server. > > > > I might even have some great improvements ready to reduce the > > testsuite time, but I'm not comfortable in committing them in this > > context. > > > > Thanks, > > Sanne > > ___ > > hibernate-dev mailing list > > hibernate-dev@lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hibernate-dev > > > > ___ > > hibernate-dev mailing list > > hibernate-dev@lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hibernate-dev > > > > > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev > ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Analyzing the ORM testsuite
I may be wrong, but I believe that the `build` task runs checkstyle, etc. Findbugs especially takes a ton of time. Have you tried just test? I have noticed that it seems like test causes javadoc to run as well, which I had intended to look at. But that seems not relevant to what you see On Thu, Apr 19, 2018, 5:01 PM Gail Badner wrote: > I'm not seeing anything unusual. I usually run `./gradlew clean test`. > > On Thu, Apr 19, 2018 at 2:20 PM, Sanne Grinovero > wrote: > > > On 19 April 2018 at 21:47, Chris Cranford wrote: > > > Sanne - > > > > > > Are you running the build task or are you executing other more specific > > > tasks and noticing the slowness? > > > > Just running "gradle clean build" from the root. I had the same > > results with OpenJDK 8 and 9, twice. Getting exactly the same failures > > as before. > > > > Thanks! > > > > > > > > On 04/19/2018 04:25 PM, Steve Ebersole wrote: > > > > > > Nothing off the top of my head. The build time is in normal range on > my > > > machine as of this morning (8 - 10 minutes). > > > > > > You mentioned a specific commit. Is that just when you noticed a > change, > > > or do you suspect something in that commit? > > > > > > I'm not at computer right now, but when I get back I will check out > that > > > commit and see if anything is strange locally. > > > > > > > > > > > > On Thu, Apr 19, 2018, 3:11 PM Sanne Grinovero > > wrote: > > > > > > Hi all, > > > > > > I'm trying to analyze the memory usage of current master of Hibernate > > > ORM; I started looking because of OOM errors on the PostgreSQL > > > testsuite (as mentioned in another thread). > > > > > > So far I've seen that Mockito is allocating a "lot of stuff"; I might > > > be able to improve some things in that area but it's a distraction > > > from my goal as clearly we use Mockito even when not running the > > > PostgreSQL backed tests and that seems to run fine. But it's polluting > > > the data reports so making it hard (and slow!) to verify if there is a > > > real issue. > > > > > > I could use some help to verify some tangential issues though; when > > > running the testsuite locally it's taking much longer than what I've > > > been used to see for the same testsuite some weeks ago, and I have > > > some failures - regularly seeing the same failures. > > > > > > These are triggered by running a default build, on H2: > > > > > > Task :hibernate-core:test > > > > > > org.hibernate.test.criteria.CriteriaLockingTest > > > > > > > testSetLockModeDifferentFromNONELogAWarnMessageWhenTheDialec > > tUseFollowOnLockingIsTrue > > > FAILED > > > org.junit.runners.model.TestTimedOutException > > > > > > org.hibernate.test.criteria.CriteriaLockingTest > > > > > > > testSetLockModeNONEDoNotLogAWarnMessageWhenTheDialectUseFoll > > owOnLockingIsTrue > > > FAILED > > > java.lang.Exception > > > > > > org.hibernate.test.criteria.CriteriaLockingTest > > > > org.hibernate.test.criteria.CriteriaLockingTest FAILED > > > java.lang.Exception > > > > > > org.hibernate.test.locking.warning.LockNoneWarmingTest > > > > org.hibernate.test.locking.warning.LockNoneWarmingTest FAILED > > > java.lang.Exception > > > > > > 6307 tests completed, 4 failed, 327 skipped > > > > > > Yet the puzzling point: on ci.hibernate.org I don't see these failures > > > happening, nor the testsuite time seems to have increased of any > > > significant amount (it regularly completes in somewhere between 15 and > > > 20 minutes). It takes me more than an hour to get this result on a > > > machine which would normally complete the testsuite in 8 minutes, and > > > I got the same 4 failures when re-running it a second time. > > > > > > Could other developers also please checkout master - specifically at > > > commit 291d4a3eeaa2ade32a42cfbcad5868b2114c34fe - and let me know if > > > it works allright for you and if the build time is consistent with the > > > usual times for your machine? I need to understand what might be > > > different between my workstation and the CI server. > > > > > > I might even have some great improvements ready to reduce the > > > testsuite time, but I'm not comfortable in committing them in this > > > context. > > > > > > Thanks, > > > Sanne > > > ___ > > > hibernate-dev mailing list > > > hibernate-dev@lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/hibernate-dev > > > > > > ___ > > > hibernate-dev mailing list > > > hibernate-dev@lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/hibernate-dev > > > > > > > > ___ > > hibernate-dev mailing list > > hibernate-dev@lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hibernate-dev > > > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev > _
Re: [hibernate-dev] Analyzing the ORM testsuite
On 19 April 2018 at 22:50, Chris Cranford wrote: > The entire build process under OpenJDK 8 here: > > BUILD SUCCESSFUL in 47m 26s > 251 actionable tasks: 233 executed, 18 up-to-date > > The only thing I noticed was that while executing the hibernate-core tests, > the memory consumption seemed quite high toward the end of the test suite, > particularly ~3.5GB memory consumed just to run the hibernate-core tests > alone and that FindBugs complained about some magic numbers on some zip > files while inspecting hibernate-osgi. > > Any idea if any substantial amount of that memory consumption related to the > test framework tracking the output/errors of those executed tests? I guess that confirms there's something specific to my environment. That's a great hint to have, thanks! Regarding the memory, yes I'm seeing the same. >90% of that is Mockito, I'm exploring some ideas to reduce its memory usage; I have some working experiments but I'm not happy with them: they make maintenance of tests error prone so tomorrow I'll try some alternatives. > > On 04/19/2018 05:20 PM, Sanne Grinovero wrote: > > On 19 April 2018 at 21:47, Chris Cranford wrote: > > Sanne - > > Are you running the build task or are you executing other more specific > tasks and noticing the slowness? > > Just running "gradle clean build" from the root. I had the same > results with OpenJDK 8 and 9, twice. Getting exactly the same failures > as before. > > Thanks! > > On 04/19/2018 04:25 PM, Steve Ebersole wrote: > > Nothing off the top of my head. The build time is in normal range on my > machine as of this morning (8 - 10 minutes). > > You mentioned a specific commit. Is that just when you noticed a change, > or do you suspect something in that commit? > > I'm not at computer right now, but when I get back I will check out that > commit and see if anything is strange locally. > > > > On Thu, Apr 19, 2018, 3:11 PM Sanne Grinovero wrote: > > Hi all, > > I'm trying to analyze the memory usage of current master of Hibernate > ORM; I started looking because of OOM errors on the PostgreSQL > testsuite (as mentioned in another thread). > > So far I've seen that Mockito is allocating a "lot of stuff"; I might > be able to improve some things in that area but it's a distraction > from my goal as clearly we use Mockito even when not running the > PostgreSQL backed tests and that seems to run fine. But it's polluting > the data reports so making it hard (and slow!) to verify if there is a > real issue. > > I could use some help to verify some tangential issues though; when > running the testsuite locally it's taking much longer than what I've > been used to see for the same testsuite some weeks ago, and I have > some failures - regularly seeing the same failures. > > These are triggered by running a default build, on H2: > > Task :hibernate-core:test > > org.hibernate.test.criteria.CriteriaLockingTest > > > testSetLockModeDifferentFromNONELogAWarnMessageWhenTheDialectUseFollowOnLockingIsTrue > FAILED > org.junit.runners.model.TestTimedOutException > > org.hibernate.test.criteria.CriteriaLockingTest > > > testSetLockModeNONEDoNotLogAWarnMessageWhenTheDialectUseFollowOnLockingIsTrue > FAILED > java.lang.Exception > > org.hibernate.test.criteria.CriteriaLockingTest > > org.hibernate.test.criteria.CriteriaLockingTest FAILED > java.lang.Exception > > org.hibernate.test.locking.warning.LockNoneWarmingTest > > org.hibernate.test.locking.warning.LockNoneWarmingTest FAILED > java.lang.Exception > > 6307 tests completed, 4 failed, 327 skipped > > Yet the puzzling point: on ci.hibernate.org I don't see these failures > happening, nor the testsuite time seems to have increased of any > significant amount (it regularly completes in somewhere between 15 and > 20 minutes). It takes me more than an hour to get this result on a > machine which would normally complete the testsuite in 8 minutes, and > I got the same 4 failures when re-running it a second time. > > Could other developers also please checkout master - specifically at > commit 291d4a3eeaa2ade32a42cfbcad5868b2114c34fe - and let me know if > it works allright for you and if the build time is consistent with the > usual times for your machine? I need to understand what might be > different between my workstation and the CI server. > > I might even have some great improvements ready to reduce the > testsuite time, but I'm not comfortable in committing them in this > context. > > Thanks, > Sanne > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev > > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev > > > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate
Re: [hibernate-dev] Analyzing the ORM testsuite
I checked - `build` does in fact pull in a bunch of tasks that you do not care about (checkstyle, findbugs, etc). The build I just ran did take longer than I have been seeing - it took 16 minutes, but that's not the magnitude you see. I mentioned performing test caused javadoc to be generated. I found the culprit for that - hibernate-orm-modules. Any time tests use hibernate-orm-modules, hibernate-orm-modules triggers a full build and publish of the project's artifacts (jar, source jar, javadoc jar). This accounts for at least some of the additional time. I think using build rather than test probably accounts for the rest On Thu, Apr 19, 2018 at 5:35 PM Sanne Grinovero wrote: > On 19 April 2018 at 22:50, Chris Cranford wrote: > > The entire build process under OpenJDK 8 here: > > > > BUILD SUCCESSFUL in 47m 26s > > 251 actionable tasks: 233 executed, 18 up-to-date > > > > The only thing I noticed was that while executing the hibernate-core > tests, > > the memory consumption seemed quite high toward the end of the test > suite, > > particularly ~3.5GB memory consumed just to run the hibernate-core tests > > alone and that FindBugs complained about some magic numbers on some zip > > files while inspecting hibernate-osgi. > > > > Any idea if any substantial amount of that memory consumption related to > the > > test framework tracking the output/errors of those executed tests? > > I guess that confirms there's something specific to my environment. > That's a great hint to have, thanks! > > Regarding the memory, yes I'm seeing the same. >90% of that is > Mockito, I'm exploring some ideas to reduce its memory usage; I have > some working experiments but I'm not happy with them: they make > maintenance of tests error prone so tomorrow I'll try some > alternatives. > > > > > > On 04/19/2018 05:20 PM, Sanne Grinovero wrote: > > > > On 19 April 2018 at 21:47, Chris Cranford wrote: > > > > Sanne - > > > > Are you running the build task or are you executing other more specific > > tasks and noticing the slowness? > > > > Just running "gradle clean build" from the root. I had the same > > results with OpenJDK 8 and 9, twice. Getting exactly the same failures > > as before. > > > > Thanks! > > > > On 04/19/2018 04:25 PM, Steve Ebersole wrote: > > > > Nothing off the top of my head. The build time is in normal range on my > > machine as of this morning (8 - 10 minutes). > > > > You mentioned a specific commit. Is that just when you noticed a change, > > or do you suspect something in that commit? > > > > I'm not at computer right now, but when I get back I will check out that > > commit and see if anything is strange locally. > > > > > > > > On Thu, Apr 19, 2018, 3:11 PM Sanne Grinovero > wrote: > > > > Hi all, > > > > I'm trying to analyze the memory usage of current master of Hibernate > > ORM; I started looking because of OOM errors on the PostgreSQL > > testsuite (as mentioned in another thread). > > > > So far I've seen that Mockito is allocating a "lot of stuff"; I might > > be able to improve some things in that area but it's a distraction > > from my goal as clearly we use Mockito even when not running the > > PostgreSQL backed tests and that seems to run fine. But it's polluting > > the data reports so making it hard (and slow!) to verify if there is a > > real issue. > > > > I could use some help to verify some tangential issues though; when > > running the testsuite locally it's taking much longer than what I've > > been used to see for the same testsuite some weeks ago, and I have > > some failures - regularly seeing the same failures. > > > > These are triggered by running a default build, on H2: > > > > Task :hibernate-core:test > > > > org.hibernate.test.criteria.CriteriaLockingTest > > > > > > testSetLockModeDifferentFromNONELogAWarnMessageWhenTheDialectUseFollowOnLockingIsTrue > > FAILED > > org.junit.runners.model.TestTimedOutException > > > > org.hibernate.test.criteria.CriteriaLockingTest > > > > > > testSetLockModeNONEDoNotLogAWarnMessageWhenTheDialectUseFollowOnLockingIsTrue > > FAILED > > java.lang.Exception > > > > org.hibernate.test.criteria.CriteriaLockingTest > > > org.hibernate.test.criteria.CriteriaLockingTest FAILED > > java.lang.Exception > > > > org.hibernate.test.locking.warning.LockNoneWarmingTest > > > org.hibernate.test.locking.warning.LockNoneWarmingTest FAILED > > java.lang.Exception > > > > 6307 tests completed, 4 failed, 327 skipped > > > > Yet the puzzling point: on ci.hibernate.org I don't see these failures > > happening, nor the testsuite time seems to have increased of any > > significant amount (it regularly completes in somewhere between 15 and > > 20 minutes). It takes me more than an hour to get this result on a > > machine which would normally complete the testsuite in 8 minutes, and > > I got the same 4 failures when re-running it a second time. > > > > Could other developers also please checkout master - specifically at > > commit 291d4a
Re: [hibernate-dev] Analyzing the ORM testsuite
The postgres build is fine again, thanks for all the help! We re-enabled the CI job. On Fri, 20 Apr 2018, 01:14 Steve Ebersole, wrote: > > I checked - `build` does in fact pull in a bunch of tasks that you do not > care about (checkstyle, findbugs, etc). > > The build I just ran did take longer than I have been seeing - it took 16 > minutes, but that's not the magnitude you see. > > I mentioned performing test caused javadoc to be generated. I found the > culprit for that - hibernate-orm-modules. Any time tests > use hibernate-orm-modules, hibernate-orm-modules triggers a full build and > publish of the project's artifacts (jar, source jar, javadoc jar). This > accounts for at least some of the additional time. I think using build > rather than test probably accounts for the rest > > > On Thu, Apr 19, 2018 at 5:35 PM Sanne Grinovero > wrote: > >> On 19 April 2018 at 22:50, Chris Cranford wrote: >> > The entire build process under OpenJDK 8 here: >> > >> > BUILD SUCCESSFUL in 47m 26s >> > 251 actionable tasks: 233 executed, 18 up-to-date >> > >> > The only thing I noticed was that while executing the hibernate-core >> tests, >> > the memory consumption seemed quite high toward the end of the test >> suite, >> > particularly ~3.5GB memory consumed just to run the hibernate-core tests >> > alone and that FindBugs complained about some magic numbers on some zip >> > files while inspecting hibernate-osgi. >> > >> > Any idea if any substantial amount of that memory consumption related >> to the >> > test framework tracking the output/errors of those executed tests? >> >> I guess that confirms there's something specific to my environment. >> That's a great hint to have, thanks! >> >> Regarding the memory, yes I'm seeing the same. >90% of that is >> Mockito, I'm exploring some ideas to reduce its memory usage; I have >> some working experiments but I'm not happy with them: they make >> maintenance of tests error prone so tomorrow I'll try some >> alternatives. >> >> >> > >> > On 04/19/2018 05:20 PM, Sanne Grinovero wrote: >> > >> > On 19 April 2018 at 21:47, Chris Cranford wrote: >> > >> > Sanne - >> > >> > Are you running the build task or are you executing other more specific >> > tasks and noticing the slowness? >> > >> > Just running "gradle clean build" from the root. I had the same >> > results with OpenJDK 8 and 9, twice. Getting exactly the same failures >> > as before. >> > >> > Thanks! >> > >> > On 04/19/2018 04:25 PM, Steve Ebersole wrote: >> > >> > Nothing off the top of my head. The build time is in normal range on my >> > machine as of this morning (8 - 10 minutes). >> > >> > You mentioned a specific commit. Is that just when you noticed a >> change, >> > or do you suspect something in that commit? >> > >> > I'm not at computer right now, but when I get back I will check out that >> > commit and see if anything is strange locally. >> > >> > >> > >> > On Thu, Apr 19, 2018, 3:11 PM Sanne Grinovero >> wrote: >> > >> > Hi all, >> > >> > I'm trying to analyze the memory usage of current master of Hibernate >> > ORM; I started looking because of OOM errors on the PostgreSQL >> > testsuite (as mentioned in another thread). >> > >> > So far I've seen that Mockito is allocating a "lot of stuff"; I might >> > be able to improve some things in that area but it's a distraction >> > from my goal as clearly we use Mockito even when not running the >> > PostgreSQL backed tests and that seems to run fine. But it's polluting >> > the data reports so making it hard (and slow!) to verify if there is a >> > real issue. >> > >> > I could use some help to verify some tangential issues though; when >> > running the testsuite locally it's taking much longer than what I've >> > been used to see for the same testsuite some weeks ago, and I have >> > some failures - regularly seeing the same failures. >> > >> > These are triggered by running a default build, on H2: >> > >> > Task :hibernate-core:test >> > >> > org.hibernate.test.criteria.CriteriaLockingTest > >> > >> > >> testSetLockModeDifferentFromNONELogAWarnMessageWhenTheDialectUseFollowOnLockingIsTrue >> > FAILED >> > org.junit.runners.model.TestTimedOutException >> > >> > org.hibernate.test.criteria.CriteriaLockingTest > >> > >> > >> testSetLockModeNONEDoNotLogAWarnMessageWhenTheDialectUseFollowOnLockingIsTrue >> > FAILED >> > java.lang.Exception >> > >> > org.hibernate.test.criteria.CriteriaLockingTest > >> > org.hibernate.test.criteria.CriteriaLockingTest FAILED >> > java.lang.Exception >> > >> > org.hibernate.test.locking.warning.LockNoneWarmingTest > >> > org.hibernate.test.locking.warning.LockNoneWarmingTest FAILED >> > java.lang.Exception >> > >> > 6307 tests completed, 4 failed, 327 skipped >> > >> > Yet the puzzling point: on ci.hibernate.org I don't see these failures >> > happening, nor the testsuite time seems to have increased of any >> > significant amount (it regularly completes in somewhere between 15 and >> > 20 minutes). It
