Hi Micheal, Thanks Dan and Michael for all your inputs.
I tried this approach as well, but didn't find any success. During unmapping i used the API munmap_flags() with "UNMAP_INIT_OPTIONAL" flag to avoid the zero initialization during the next mmaping. Another thought I have here is that the old mmapped regions may not be initialized with zeros, but instead the regions are getting synced with the data in disk file (which is full of zeros). Not sure if can happen with mmap() API though? Thanks, Praveen -----Original Message----- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Black, Michael (IS) Sent: Thursday, October 14, 2010 5:21 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] TestFixture 3.7.2 - Some WAL tests fail on QNX OS I sent this before...have you tried this? According to the QNX mmap page http://www.qnx.com/developers/docs/6.3.0SP3/neutrino/lib_ref/m/mmap.html <https://owa1.ngc.com/exchweb/bin/redir.asp?URL=http://www.qnx.com/developers/docs/6.3.0SP3/neutrino/lib_ref/m/mmap.html> MAP_NOINIT When specified, the POSIX requirement that the memory be zeroed is relaxed. The physical memory being used for this allocation must have been previously freed with UNMAP_INIT_OPTIONAL for this flag to have any effect. ________________________________ This flag was added in the QNX Neutrino Core OS 6.3.2. Interesting that this claims it's a POSIX requirement but I don't think most any others do this. Michael D. Black Senior Scientist Advanced Analytics Directorate Northrop Grumman Information Systems ________________________________ From: sqlite-users-boun...@sqlite.org on behalf of Raj, Praveen Sent: Thu 10/14/2010 5:53 AM To: General Discussion of SQLite Database Subject: EXTERNAL:Re: [sqlite] TestFixture 3.7.2 - Some WAL tests fail on QNX OS Hi Dan, I did some more investigation on the issue and i feel there is synchronization problem happening here. After mmapping the shm (wal index) file to process memory, the WAL indexes are written into the mmapped area, and this data is not getting synchronized with physical (shm) file. As a result when the mmap() function is called the second time to map the 32k-64k memory region, it is synchronizing the complete mmapped region (previous 32k regions) with physical file, even though a valid offset is passed. Not sure if this is the actual behaviour of mmap() call. While debugging, before the mmap() call i checked mmapped region and it had valid indexes, whereas after the call all became 0's. Also i found that the shm file is always filled with 0's even after commits. When i added the msync() statement (to sync the shm file) before mmap call as shown below, the problem is not seen. In this case the shm file has valid 32-bit indexes, as data is synchronized before next mmap call is executed. while(pShmNode->nRegion<=iRegion){ int ret = msync( apNew[0], iRegion*szRegion, MS_SYNC); void *pMem = mmap(0, szRegion, PROT_READ|PROT_WRITE, MAP_SHARED, pShmNode->h, pShmNode->nRegion*szRegion ); With the above msync() call all my failed test cases are passing. I don't see any msync() call in the SQLite amalgamation/wal.c file. I believe the data in mapped region and physical file are not synched automatically. We need to explicitly do it using msync() call. Don't know if there is any other mechanism in SQLite through which the data is synchronized. Does the call to sqlite3OsSync() sync the shm file as well? or is the shm file not syned purposefully? This is all my understanding and not sure if this is causing the actual issue. Please guide me if my approach/understanding is incorrect. Thanks, Praveen -----Original Message----- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Dan Kennedy Sent: Friday, October 08, 2010 9:33 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] TestFixture 3.7.2 - Some WAL tests fail on QNX OS On Oct 8, 2010, at 9:44 PM, Raj, Praveen wrote: > Hello, > > I debugged the SQLite functions and here is my finding: > > The call to "mmap" in the function "unixShmMap" is causing the issue. > void *pMem = mmap(0, szRegion, PROT_READ|PROT_WRITE, > MAP_SHARED, pShmNode->h, iRegion*szRegion); > > It is setting the previous memory region/regions to zero while mapping > the new ones. Mmap call internally uses the QNX API mmap64() to map > the required memory region. Not sure on what is happening here. Just > need to dig into memory mapping to find whats happening and hopefully > find a solution. > > Dan - Do you have any idea on why this could be happening? Sounds like a bug in QNX to me. _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users The information contained in this message may be confidential and legally protected under applicable law. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, forwarding, dissemination, or reproduction of this message is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message. _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
