Re: [Gluster-devel] glusterfs-3.5.0beta4 released
Memory usage seems better than in beta3. I have been building software in a glusterfs volume for a few hours, and glusterfsd memory footprint is still reasonable. And it did not crash. There have been fixes checked in on that front, or I am just lucky? -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz m...@netbsd.org ___ Gluster-devel mailing list Gluster-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] glusterfs-3.5.0beta4 released
On 03/09/2014 02:39 PM, Emmanuel Dreyfus wrote: Memory usage seems better than in beta3. I have been building software in a glusterfs volume for a few hours, and glusterfsd memory footprint is still reasonable. And it did not crash. There have been fixes checked in on that front, or I am just lucky? Do not observe any fixes related to memory leaks. Would it be possible to get a valgrind report for the glusterfsd processes if you happen to observe the leak again? Thanks, Vijay ___ Gluster-devel mailing list Gluster-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] glusterfs-3.5.0beta4 released
Vijay Bellur wrote: > Do not observe any fixes related to memory leaks. Would it be possible > to get a valgrind report for the glusterfsd processes if you happen to > observe the leak again? Unfortunately, valgrind was not ported on NetBSD. -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz m...@netbsd.org ___ Gluster-devel mailing list Gluster-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] glusterfs-3.5.0beta4 released
Vijay Bellur wrote: > Do not observe any fixes related to memory leaks. Would it be possible > to get a valgrind report for the glusterfsd processes if you happen to > observe the leak again? I have the answer: I made a mistake reinstalling, and I am now running 3.5.0qa2 instead of 3.5.0beta4 That is not good news. -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz m...@netbsd.org ___ Gluster-devel mailing list Gluster-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] GlusterFS unit test framework
On 06/03/2014, at 9:01 PM, Luis Pabon wrote: > Hi all, >The patch is now live in GlusterFS. This patch will allow you to write > unit tests for your new patches or if you want, you can start writing unit > tests for older code. Writing unit tests for older code is a great way to > learn the code (which I will be doing :-)). I have also created a > presentation to help explain what unit tests are: > > http://slides-lpabon.rhcloud.com/feb24_glusterfs_unittest.html#/ > > The framework will also help in situations where developers are depending on > APIs from other contributors. In this situation, the developer can write > their patch and test their code by mocking any APIs their code is dependent > on. This method allows for parallel development without the need to wait for > the APIs to be available. It turns out this patch is breaking the build on EL5. :( Looking through the docs on your cmockery2 GitHub repo, it's hard to tell which platforms are supported. Should it work on EL5? :) Regards and best wishes, Justin Clift -- Open Source and Standards @ Red Hat twitter.com/realjustinclift ___ Gluster-devel mailing list Gluster-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] glusterfs-3.5.0beta4 released
On 03/09/2014 04:17 PM, Emmanuel Dreyfus wrote: Vijay Bellur wrote: Do not observe any fixes related to memory leaks. Would it be possible to get a valgrind report for the glusterfsd processes if you happen to observe the leak again? Unfortunately, valgrind was not ported on NetBSD. In that case, we can look at utilizing the built in memory accounting provided by glusterfs. Steps for that would include: 1. gluster volume set config.memory-accounting on 2. (Re)start the volume 3. After the memory consumption increases, trigger a statedump by sending SIGUSR1 to the brick process. 4. statedump file would be created at $(INSTALL_PREFIX)/var/run/gluster/ 5. grep -A4 memusage 6. Noticing those allocations which have significant size & num_allocs in the output of grep can help in narrowing down the leak. (usage-type can be looked up from mem-types.h in libglusterfs and -mem-types.h located in the respective translator directory). - Vijay ___ Gluster-devel mailing list Gluster-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] GlusterFS unit test framework
Hi Justin, Do you have the logs for the build, or how to reproduce it? Let's take this offline and see what we can do. - Luis On 03/09/2014 09:18 AM, Justin Clift wrote: On 06/03/2014, at 9:01 PM, Luis Pabon wrote: Hi all, The patch is now live in GlusterFS. This patch will allow you to write unit tests for your new patches or if you want, you can start writing unit tests for older code. Writing unit tests for older code is a great way to learn the code (which I will be doing :-)). I have also created a presentation to help explain what unit tests are: http://slides-lpabon.rhcloud.com/feb24_glusterfs_unittest.html#/ The framework will also help in situations where developers are depending on APIs from other contributors. In this situation, the developer can write their patch and test their code by mocking any APIs their code is dependent on. This method allows for parallel development without the need to wait for the APIs to be available. It turns out this patch is breaking the build on EL5. :( Looking through the docs on your cmockery2 GitHub repo, it's hard to tell which platforms are supported. Should it work on EL5? :) Regards and best wishes, Justin Clift -- Open Source and Standards @ Red Hat twitter.com/realjustinclift ___ Gluster-devel mailing list Gluster-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] glusterfs-3.5.0beta4 released
Emmanuel Dreyfus wrote: > > Do not observe any fixes related to memory leaks. Would it be possible > > to get a valgrind report for the glusterfsd processes if you happen to > > observe the leak again? > > I have the answer: I made a mistake reinstalling, and I am now running > 3.5.0qa2 instead of 3.5.0beta4 And there is another reason: I forgot the "replica 2" when I created the volume, it was therefore not using AFR. That may help finding where the issue is. Now running 3.5.0beta4 on a 2x2 replicated/distributed volume. Just unpacking a tar archive already gives an impressive result: glusterfsd brick processes get a 1.6 GB memory footprint. PID USER PRI NICE SIZE RES STATE TIME WCPU CPU COMMAND 619 root 840 1635M 1574M select/1 33:08 2.34% 2.34% glusterfsd -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz m...@netbsd.org ___ Gluster-devel mailing list Gluster-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] glusterfs-3.5.0beta4 released
Vijay Bellur wrote: > 6. Noticing those allocations which have significant size & num_allocs > in the output of grep can help in narrowing down the leak. Here are the worst offenders for num_allocs: [protocol/server.gfs35b4-server - usage-type 48 memusage] size=17344060 num_allocs=408214 max_size=17344112 max_num_allocs=408214 total_allocs=621264 [protocol/server.gfs35b4-server - usage-type 45 memusage] size=5236270 num_allocs=219576 max_size=5238059 max_num_allocs=219577 total_allocs=476119 [protocol/server.gfs35b4-server - usage-type 38 memusage] size=635362 num_allocs=219566 max_size=635362 max_num_allocs=219566 total_allocs=275956 [features/access-control.gfs35b4-access-control - usage-type 48 memusage] size=3496124 num_allocs=87377 max_size=3496124 max_num_allocs=87377 total_allocs=87377 [features/access-control.gfs35b4-access-control - usage-type 39 memusage] size=93236 num_allocs=46618 max_size=93236 max_num_allocs=46618 total_allocs=46618 [features/access-control.gfs35b4-access-control - usage-type 45 memusage] size=1142043 num_allocs=46614 max_size=1142043 max_num_allocs=46614 total_allocs=46614 I also have this with big size: [global.glusterfs - usage-type 49 memusage] size=9711104 num_allocs=9 max_size=9711104 max_num_allocs=9 total_allocs=9 [protocol/server.gfs35b4-server - usage-type 49 memusage] size=5525504 num_allocs=4 max_size=5525504 max_num_allocs=4 total_allocs=4 -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz m...@netbsd.org ___ Gluster-devel mailing list Gluster-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/gluster-devel
