On 06/01/2011 11:05 PM, william felipe_welter wrote: > I recompile my kernel without hugetlb .. and the result are the same.. > > My test program still resulting: > PATH=/dev/shm/teste123XXXXXX > page size=20000 > fd=3 > ADDR_ORIG:0xe000a000 ADDR:0xffffffff > Erro > > And Pacemaker still resulting because the mmap error: > Could not initialize Cluster Configuration Database API instance error 2 >
Give the patch I posted recently a spin - corosync WFM with this patch on sparc64 with hugetlb set. Please report back results. Regards -steve > For make sure that i have disable the hugetlb there is my /proc/meminfo: > MemTotal: 33093488 kB > MemFree: 32855616 kB > Buffers: 5600 kB > Cached: 53480 kB > SwapCached: 0 kB > Active: 45768 kB > Inactive: 28104 kB > Active(anon): 18024 kB > Inactive(anon): 1560 kB > Active(file): 27744 kB > Inactive(file): 26544 kB > Unevictable: 0 kB > Mlocked: 0 kB > SwapTotal: 6104680 kB > SwapFree: 6104680 kB > Dirty: 0 kB > Writeback: 0 kB > AnonPages: 14936 kB > Mapped: 7736 kB > Shmem: 4624 kB > Slab: 39184 kB > SReclaimable: 10088 kB > SUnreclaim: 29096 kB > KernelStack: 7088 kB > PageTables: 1160 kB > Quicklists: 17664 kB > NFS_Unstable: 0 kB > Bounce: 0 kB > WritebackTmp: 0 kB > CommitLimit: 22651424 kB > Committed_AS: 519368 kB > VmallocTotal: 1069547520 kB > VmallocUsed: 11064 kB > VmallocChunk: 1069529616 kB > > > 2011/6/1 Steven Dake <sd...@redhat.com>: >> On 06/01/2011 07:42 AM, william felipe_welter wrote: >>> Steven, >>> >>> cat /proc/meminfo >>> ... >>> HugePages_Total: 0 >>> HugePages_Free: 0 >>> HugePages_Rsvd: 0 >>> HugePages_Surp: 0 >>> Hugepagesize: 4096 kB >>> ... >>> >> >> It definitely requires a kernel compile and setting the config option to >> off. I don't know the debian way of doing this. >> >> The only reason you may need this option is if you have very large >> memory sizes, such as 48GB or more. >> >> Regards >> -steve >> >>> Its 4MB.. >>> >>> How can i disable hugetlb ? ( passing CONFIG_HUGETLBFS=n at boot to >>> kernel ?) >>> >>> 2011/6/1 Steven Dake <sd...@redhat.com <mailto:sd...@redhat.com>> >>> >>> On 06/01/2011 01:05 AM, Steven Dake wrote: >>> > On 05/31/2011 09:44 PM, Angus Salkeld wrote: >>> >> On Tue, May 31, 2011 at 11:52:48PM -0300, william felipe_welter >>> wrote: >>> >>> Angus, >>> >>> >>> >>> I make some test program (based on the code coreipcc.c) and i >>> now i sure >>> >>> that are problems with the mmap systems call on sparc.. >>> >>> >>> >>> Source code of my test program: >>> >>> >>> >>> #include <stdlib.h> >>> >>> #include <sys/mman.h> >>> >>> #include <stdio.h> >>> >>> >>> >>> #define PATH_MAX 36 >>> >>> >>> >>> int main() >>> >>> { >>> >>> >>> >>> int32_t fd; >>> >>> void *addr_orig; >>> >>> void *addr; >>> >>> char path[PATH_MAX]; >>> >>> const char *file = "teste123XXXXXX"; >>> >>> size_t bytes=10024; >>> >>> >>> >>> snprintf (path, PATH_MAX, "/dev/shm/%s", file); >>> >>> printf("PATH=%s\n",path); >>> >>> >>> >>> fd = mkstemp (path); >>> >>> printf("fd=%d \n",fd); >>> >>> >>> >>> >>> >>> addr_orig = mmap (NULL, bytes, PROT_NONE, >>> >>> MAP_ANONYMOUS | MAP_PRIVATE, -1, 0); >>> >>> >>> >>> >>> >>> addr = mmap (addr_orig, bytes, PROT_READ | PROT_WRITE, >>> >>> MAP_FIXED | MAP_SHARED, fd, 0); >>> >>> >>> >>> printf("ADDR_ORIG:%p ADDR:%p\n",addr_orig,addr); >>> >>> >>> >>> >>> >>> if (addr != addr_orig) { >>> >>> printf("Erro"); >>> >>> } >>> >>> } >>> >>> >>> >>> Results on x86: >>> >>> PATH=/dev/shm/teste123XXXXXX >>> >>> fd=3 >>> >>> ADDR_ORIG:0x7f867d8e6000 ADDR:0x7f867d8e6000 >>> >>> >>> >>> Results on sparc: >>> >>> PATH=/dev/shm/teste123XXXXXX >>> >>> fd=3 >>> >>> ADDR_ORIG:0xf7f72000 ADDR:0xffffffff >>> >> >>> >> Note: 0xffffffff == MAP_FAILED >>> >> >>> >> (from man mmap) >>> >> RETURN VALUE >>> >> On success, mmap() returns a pointer to the mapped area. On >>> >> error, the value MAP_FAILED (that is, (void *) -1) is >>> returned, >>> >> and errno is set appropriately. >>> >> >>> >>> >>> >>> >>> >>> But im wondering if is really needed to call mmap 2 times ? >>> What are the >>> >>> reason to call the mmap 2 times, on the second time using the >>> address of the >>> >>> first? >>> >>> >>> >>> >>> >> Well there are 3 calls to mmap() >>> >> 1) one to allocate 2 * what you need (in pages) >>> >> 2) maps the first half of the mem to a real file >>> >> 3) maps the second half of the mem to the same file >>> >> >>> >> The point is when you write to an address over the end of the >>> >> first half of memory it is taken care of the the third mmap which >>> maps >>> >> the address back to the top of the file for you. This means you >>> >> don't have to worry about ringbuffer wrapping which can be a >>> headache. >>> >> >>> >> -Angus >>> >> >>> > >>> > interesting this mmap operation doesn't work on sparc linux. >>> > >>> > Not sure how I can help here - Next step would be a follow up with the >>> > sparc linux mailing list. I'll do that and cc you on the message >>> - see >>> > if we get any response. >>> > >>> > http://vger.kernel.org/vger-lists.html >>> > >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> 2011/5/31 Angus Salkeld <asalk...@redhat.com >>> <mailto:asalk...@redhat.com>> >>> >>> >>> >>>> On Tue, May 31, 2011 at 06:25:56PM -0300, william felipe_welter >>> wrote: >>> >>>>> Thanks Steven, >>> >>>>> >>> >>>>> Now im try to run on the MCP: >>> >>>>> - Uninstall the pacemaker 1.0 >>> >>>>> - Compile and install 1.1 >>> >>>>> >>> >>>>> But now i have problems to initialize the pacemakerd: Could not >>> >>>> initialize >>> >>>>> Cluster Configuration Database API instance error 2 >>> >>>>> Debbuging with gdb i see that the error are on the confdb.. most >>> >>>> specificaly >>> >>>>> the errors start on coreipcc.c at line: >>> >>>>> >>> >>>>> >>> >>>>> 448 if (addr != addr_orig) { >>> >>>>> 449 goto error_close_unlink; <- enter here >>> >>>>> 450 } >>> >>>>> >>> >>>>> Some ideia about what can cause this ? >>> >>>>> >>> >>>> >>> >>>> I tried porting a ringbuffer (www.libqb.org >>> <http://www.libqb.org>) to sparc and had the same >>> >>>> failure. >>> >>>> There are 3 mmap() calls and on sparc the third one keeps failing. >>> >>>> >>> >>>> This is a common way of creating a ring buffer, see: >>> >>>> >>> >>> http://en.wikipedia.org/wiki/Circular_buffer#Exemplary_POSIX_Implementation >>> >>>> >>> >>>> I couldn't get it working in the short time I tried. It's probably >>> >>>> worth looking at the clib implementation to see why it's failing >>> >>>> (I didn't get to that). >>> >>>> >>> >>>> -Angus >>> >>>> >>> >>> Note, we sorted this out we believe. Your kernel has hugetlb enabled, >>> probably with 4MB pages. This requires corosync to allocate 4MB pages. >>> >>> Can you verify your hugetlb settings? >>> >>> If you can turn this option off, you should have atleast a working >>> corosync. >>> >>> Regards >>> -steve >>> >>>> >>> >>>> _______________________________________________ >>> >>>> Pacemaker mailing list: pacema...@oss.clusterlabs.org >>> <mailto:pacema...@oss.clusterlabs.org> >>> >>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker >>> >>>> >>> >>>> Project Home: http://www.clusterlabs.org >>> >>>> Getting started: >>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf >>> >>>> Bugs: >>> >>>> >>> >>> http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker >>> >>>> >>> >>> >>> >>> >>> >>> >>> >>> -- >>> >>> William Felipe Welter >>> >>> ------------------------------ >>> >>> Consultor em Tecnologias Livres >>> >>> william.wel...@4linux.com.br <mailto:william.wel...@4linux.com.br> >>> >>> www.4linux.com.br <http://www.4linux.com.br> >>> >> >>> >>> _______________________________________________ >>> >>> Openais mailing list >>> >>> Openais@lists.linux-foundation.org >>> <mailto:Openais@lists.linux-foundation.org> >>> >>> https://lists.linux-foundation.org/mailman/listinfo/openais >>> >> >>> >> >>> >> _______________________________________________ >>> >> Pacemaker mailing list: pacema...@oss.clusterlabs.org >>> <mailto:pacema...@oss.clusterlabs.org> >>> >> http://oss.clusterlabs.org/mailman/listinfo/pacemaker >>> >> >>> >> Project Home: http://www.clusterlabs.org >>> >> Getting started: >>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf >>> >> Bugs: >>> >>> http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker >>> > >>> > _______________________________________________ >>> > Openais mailing list >>> > Openais@lists.linux-foundation.org >>> <mailto:Openais@lists.linux-foundation.org> >>> > https://lists.linux-foundation.org/mailman/listinfo/openais >>> >>> >>> _______________________________________________ >>> Pacemaker mailing list: pacema...@oss.clusterlabs.org >>> <mailto:pacema...@oss.clusterlabs.org> >>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker >>> >>> Project Home: http://www.clusterlabs.org >>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf >>> Bugs: >>> >>> http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker >>> >>> >>> >>> >>> -- >>> William Felipe Welter >>> ------------------------------ >>> Consultor em Tecnologias Livres >>> william.wel...@4linux.com.br <mailto:william.wel...@4linux.com.br> >>> www.4linux.com.br <http://www.4linux.com.br> >>> >>> >>> >>> _______________________________________________ >>> Pacemaker mailing list: pacema...@oss.clusterlabs.org >>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker >>> >>> Project Home: http://www.clusterlabs.org >>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf >>> Bugs: >>> http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker >> >> > > > _______________________________________________ Openais mailing list Openais@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/openais