We discussed reducing the number of volumes in the maintainers' meeting.Should we still go ahead and do that?
On 9 July 2018 at 15:45, Xavi Hernandez <jaher...@redhat.com> wrote: > On Mon, Jul 9, 2018 at 11:14 AM Karthik Subrahmanya <ksubr...@redhat.com> > wrote: > >> Hi Deepshikha, >> >> Are you looking into this failure? I can still see this happening for all >> the regression runs. >> > > I've executed the failing script on my laptop and all tests finish > relatively fast. What seems to take time is the final cleanup. I can see > 'semanage' taking some CPU during destruction of volumes. The test required > 350 seconds to finish successfully. > > Not sure what caused the cleanup time to increase, but I've created a bug > [1] to track this and a patch [2] to give more time to this test. This > should allow all blocked regressions to complete successfully. > > Xavi > > [1] https://bugzilla.redhat.com/show_bug.cgi?id=1599250 > [2] https://review.gluster.org/20482 > > >> Thanks & Regards, >> Karthik >> >> On Sun, Jul 8, 2018 at 7:18 AM Atin Mukherjee <amukh...@redhat.com> >> wrote: >> >>> https://build.gluster.org/job/regression-test-with- >>> multiplex/794/display/redirect has the same test failing. Is the reason >>> of the failure different given this is on jenkins? >>> >>> On Sat, 7 Jul 2018 at 19:12, Deepshikha Khandelwal <dkhan...@redhat.com> >>> wrote: >>> >>>> Hi folks, >>>> >>>> The issue[1] has been resolved. Now the softserve instance will be >>>> having 2GB RAM i.e. same as that of the Jenkins builder's sizing >>>> configurations. >>>> >>>> [1] https://github.com/gluster/softserve/issues/40 >>>> >>>> Thanks, >>>> Deepshikha Khandelwal >>>> >>>> On Fri, Jul 6, 2018 at 6:14 PM, Karthik Subrahmanya < >>>> ksubr...@redhat.com> wrote: >>>> > >>>> > >>>> > On Fri 6 Jul, 2018, 5:18 PM Deepshikha Khandelwal, < >>>> dkhan...@redhat.com> >>>> > wrote: >>>> >> >>>> >> Hi Poornima/Karthik, >>>> >> >>>> >> We've looked into the memory error that this softserve instance have >>>> >> showed up. These machine instances have 1GB RAM which is not in the >>>> >> case with the Jenkins builder. It's 2GB RAM there. >>>> >> >>>> >> We've created the issue [1] and will solve it sooner. >>>> > >>>> > Great. Thanks for the update. >>>> >> >>>> >> >>>> >> Sorry for the inconvenience. >>>> >> >>>> >> [1] https://github.com/gluster/softserve/issues/40 >>>> >> >>>> >> Thanks, >>>> >> Deepshikha Khandelwal >>>> >> >>>> >> On Fri, Jul 6, 2018 at 3:44 PM, Karthik Subrahmanya < >>>> ksubr...@redhat.com> >>>> >> wrote: >>>> >> > Thanks Poornima for the analysis. >>>> >> > Can someone work on fixing this please? >>>> >> > >>>> >> > ~Karthik >>>> >> > >>>> >> > On Fri, Jul 6, 2018 at 3:17 PM Poornima Gurusiddaiah >>>> >> > <pguru...@redhat.com> >>>> >> > wrote: >>>> >> >> >>>> >> >> The same test case is failing for my patch as well [1]. I >>>> requested for >>>> >> >> a >>>> >> >> regression system and tried to reproduce it. >>>> >> >> From my analysis, the brick process (mutiplexed) is consuming a >>>> lot of >>>> >> >> memory, and is being OOM killed. The regression has 1GB ram and >>>> the >>>> >> >> process >>>> >> >> is consuming more than 1GB. 1GB for 120 bricks is acceptable >>>> >> >> considering >>>> >> >> there is 1000 threads in that brick process. >>>> >> >> Ways to fix: >>>> >> >> - Increase the regression system RAM size OR >>>> >> >> - Decrease the number of volumes in the test case. >>>> >> >> >>>> >> >> But what is strange is why the test passes sometimes for some >>>> patches. >>>> >> >> There could be some bug/? in memory consumption. >>>> >> >> >>>> >> >> Regards, >>>> >> >> Poornima >>>> >> >> >>>> >> >> >>>> >> >> On Fri, Jul 6, 2018 at 2:11 PM, Karthik Subrahmanya >>>> >> >> <ksubr...@redhat.com> >>>> >> >> wrote: >>>> >> >>> >>>> >> >>> Hi, >>>> >> >>> >>>> >> >>> $subject is failing on centos regression for most of the patches >>>> with >>>> >> >>> timeout error. >>>> >> >>> >>>> >> >>> 07:32:34 >>>> >> >>> >>>> >> >>> ============================================================ >>>> ==================== >>>> >> >>> 07:32:34 [07:33:05] Running tests in file >>>> >> >>> ./tests/bugs/core/bug-1432542-mpx-restart-crash.t >>>> >> >>> 07:32:34 Timeout set is 300, default 200 >>>> >> >>> 07:37:34 ./tests/bugs/core/bug-1432542-mpx-restart-crash.t >>>> timed out >>>> >> >>> after 300 seconds >>>> >> >>> 07:37:34 ./tests/bugs/core/bug-1432542-mpx-restart-crash.t: bad >>>> status >>>> >> >>> 124 >>>> >> >>> 07:37:34 >>>> >> >>> 07:37:34 ********************************* >>>> >> >>> 07:37:34 * REGRESSION FAILED * >>>> >> >>> 07:37:34 * Retrying failed tests in case * >>>> >> >>> 07:37:34 * we got some spurious failures * >>>> >> >>> 07:37:34 ********************************* >>>> >> >>> 07:37:34 >>>> >> >>> 07:42:34 ./tests/bugs/core/bug-1432542-mpx-restart-crash.t >>>> timed out >>>> >> >>> after 300 seconds >>>> >> >>> 07:42:34 End of test ./tests/bugs/core/bug-1432542- >>>> mpx-restart-crash.t >>>> >> >>> 07:42:34 >>>> >> >>> >>>> >> >>> ============================================================ >>>> ==================== >>>> >> >>> >>>> >> >>> Can anyone take a look? >>>> >> >>> >>>> >> >>> Thanks, >>>> >> >>> Karthik >>>> >> >>> >>>> >> >>> >>>> >> >>> >>>> >> >>> _______________________________________________ >>>> >> >>> Gluster-devel mailing list >>>> >> >>> gluster-de...@gluster.org >>>> >> >>> https://lists.gluster.org/mailman/listinfo/gluster-devel >>>> >> >> >>>> >> >> >>>> >> > >>>> >> > _______________________________________________ >>>> >> > Gluster-infra mailing list >>>> >> > Gluster-infra@gluster.org >>>> >> > https://lists.gluster.org/mailman/listinfo/gluster-infra >>>> _______________________________________________ >>>> Gluster-devel mailing list >>>> gluster-de...@gluster.org >>>> https://lists.gluster.org/mailman/listinfo/gluster-devel >>>> >>> -- >>> - Atin (atinm) >>> >> _______________________________________________ >> Gluster-infra mailing list >> Gluster-infra@gluster.org >> https://lists.gluster.org/mailman/listinfo/gluster-infra > > > _______________________________________________ > Gluster-devel mailing list > gluster-de...@gluster.org > https://lists.gluster.org/mailman/listinfo/gluster-devel >
_______________________________________________ Gluster-infra mailing list Gluster-infra@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-infra