HI, Becky I tried the command option "-a 0 -n 0" and restart the client/server, but the same issue persists. pvfs2-ping command shows one metadata server and 4 data servers. I ran my test program on the metadata server.
meta servers: tcp://bigdata:3334 Ok data servers: tcp://bigdata:3334 Ok tcp://bigdata1:3334 Ok tcp://bigdata2:3334 Ok tcp://bigdata3:3334 Ok Wei-keng On Mar 16, 2015, at 8:26 AM, Becky Ligon wrote: > Caching is still an issue if you have servers on more than one machine and > those servers provide metadata. Even in a one-server environment, it could > make a difference. > > The "ls" command uses the kernel module and client core, which in turn use > the caches, while the pvfs2-ls command does not. > > If you don't have the appropriate sudo permissions to modify the /proc > filesystem, then you can start the client with the caches turned off. > > Example: > > ./pvfs2-client -a 0 -n 0 > > > If you execute pvfs2-client --help, you will see these options. > > > Becky > > Sent from my iPhone > >> On Mar 15, 2015, at 5:05 PM, Wei-keng Liao <[email protected]> >> wrote: >> >> I assume after 60 seconds, the client will flush the cache. >> Please note I am running orangefs client and server on the same machine. >> In this case, should caching become an issue? >> >> Long after 60 seconds of the file creation, command ls still could not find >> the file. >> >> I got permission denied when running the two echo commands you suggested. >> I DO have sudo permission. I also tried vi those files but got error of >> "/proc/sys/pvfs2/acache/timeout-msecs" E667: Fsync failed >> >> Also, how do I set this automatically after system reboot? >> >> Wei-keng >> >>> On Mar 15, 2015, at 2:16 PM, Becky Ligon wrote: >>> >>> Wei-keng: >>> >>> This is most likely a caching issue with the client. By default, we set >>> the cache to timeout after 60 seconds, which may be too high in your >>> environment. Or, you have deleted and redefined a file using the same name >>> outside of the client where you are seeing the question marks, in which >>> case, the cache would be wrong for that file. >>> >>> To verify, turn off caching to see if this resolves your problem: >>> >>> As root on your client machine: >>> >>> echo "0" > /proc/sys/pvfs2/acache/timeout-msecs >>> echo "0" > /proc/sys/pvfs2/ncache/timeout-msecs >>> >>> If this change fixes your problem, try setting the timeout-msecs to >>> something more appropriate for your environment. >>> >>> Becky >>> >>> On Sun, Mar 15, 2015 at 11:43 AM, Wei-keng Liao >>> <[email protected]> wrote: >>> Hi >>> >>> I am having problems with OrangeFS 2.9.1 and MPICH 3.1.4. >>> >>> Here is my system settings: >>> Linux Kernel 2.6.32 >>> Berkeley DB version 6.1.19 >>> >>> A simple MPI test program that calls MPI_File_open and MPI_File_write_all >>> is used and ran two processes on the same host. >>> >>> The MPI commands I used with prefix file names to force ADIO drivers: >>> mpiexec -n 2 coll_write /orangefs/wkliao/testfile >>> mpiexec -n 2 coll_write pvfs2:/orangefs/wkliao/testfile.pvfs2 >>> mpiexec -n 2 coll_write ufs:/orangefs/wkliao/testfile.ufs >>> The first two will use the pvfs2 driver and the 3rd the ufs driver. >>> >>> Here is what I see when running "ls -l" and "pvfs2-ls -l" commands. >>> >>> % ls -l /orangefs/wkliao/ >>> ls: cannot access /orangefs/wkliao/testfile: No such file or directory >>> ls: cannot access /orangefs/wkliao/testfile.pvfs2: No such file or directory >>> total 31252 >>> ?????????? ? ? ? ? ? testfile >>> ?????????? ? ? ? ? ? testfile.pvfs2 >>> -rw------- 1 wkliao users 32000000 Mar 13 18:55 testfile.ufs >>> >>> % pvfs2-ls -l /orangefs/wkliao/ >>> -rw-r--r-- 1 wkliao users 31000000 2015-03-13 18:55 testfile >>> -rw------- 1 wkliao users 32000000 2015-03-13 18:55 testfile.ufs >>> -rw-r--r-- 1 wkliao users 31000000 2015-03-13 18:55 >>> testfile.pvfs2 >>> >>> My config.log file for building orangefs can be found in this URL >>> http://www.ece.northwestern.edu/~wkliao/config.log >>> >>> Wei-keng >>> >>> _______________________________________________ >>> Pvfs2-users mailing list >>> [email protected] >>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users >> _______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
