Re: [OpenAFS-devel] Cache inconsistency in client 1.4.8 and above

[Felix Frank]

On Fri, 17 Apr 2009, Felix Frank wrote:

On Thu, 16 Apr 2009, Marc Dionne wrote:
On 04/16/2009 08:25 AM, Felix Frank wrote:
-    if (!avc->states & CPageWrite)

I see a bug there - this line probably wants to be:
   if (!(avc->states & CPageWrite))

So the recursion was avoided by never actually doing anything in 
StoreAllSegments, since CPageWrite never got set and the condition was 
always false.

I guess this explains why mmap was severely broken since 1.4.8

That's not all - I retried mmap_test using vanilla 1.4.10. A file of 600MB
with a 64MB disk cache is corrupted starting somewhere above 80% for me.

My suspicion is that much data never gets written to the cache (I could 
observe something like that during testing my alternative hack). Some data 
can be read OK by the local client (maybe because the local file system 
cache has it still available?)

The matter is different for multiple clients and (apparently) cache 
manager restarts before reading (but mmap_test cannot verify either).

With the fix above, my larger mmap test quickly runs into a deadlock again. 
Looks like cache_write_pages is trying to lock the page that is currently 
being written:

I think I just reproduced :/

Even misbehave.c deadlocks the client with your fix in place (and that is
writing one byte). Are we sure the code that handles CPageWrite in
afs/LINUX/osi_vnodeops.c does what it's supposed to do?#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/mman.h>
#include <unistd.h>
#include <fcntl.h>

int main(int argc, char **argv)
{
    char *file = "mapped-file.bin";
    char *map = NULL;
    int fd;

    if ( argc > 1 )
        file = argv[1];

    printf("Using file %s...\n", file);

    fd = open(file, O_RDWR | O_CREAT);
    if ( fd == -1 ) {
        perror(file);
        return 1;
    }

    write(fd, "1\n", 2);

    if ( (map = (char*)mmap(NULL, 1, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0)) 
                        == (char*) -1 ) {
        perror("mmap");
        return 1;
    }

    close(fd);

    printf("Mapped and closed %s, first byte is %u...\n", file, map[0]);

    map[0]++;

    printf("Changed first byte to %u, unmapping...\n", map[0]);

    munmap((void*)map, 1);

    return 0;
}