The requested 2 files can be found in the following URLs. http://www.ece.northwestern.edu/~wkliao/config.log http://www.ece.northwestern.edu/~wkliao/pvfs2-server
Wei-keng On Mar 16, 2015, at 1:29 PM, Becky Ligon wrote: > Can you also send me your config.log file from when you compiled the source? > > Becky > > On Mon, Mar 16, 2015 at 2:28 PM, Becky Ligon <[email protected]> wrote: > If you are experimenting with OrangeFS, then having one metadata and 4 data > servers is fine. > > Can you send me your pvfs2-server init file, the one used with the > /sbin/service command? > > Becky > > On Mon, Mar 16, 2015 at 1:27 PM, Wei-keng Liao <[email protected]> > wrote: > On Mar 16, 2015, at 12:02 PM, Becky Ligon wrote: > > > Wei-Keing: > > > > Did you umount and mount the filesystem? If not, umount the filesystem, > > restart the client core, and then mount the filesystem again. > > Yes. My restart command ran "/sbin/servive pvfs2-server restart" > the script contains both client and server commands, include client's umount > and mount. > > > > I also suggest that you define your environment so that each of your > > machines (bigdata, bigdata1, bigdata2, bigdata3) have their pvfs servers > > configured to handle both I/O and metadata. To do this, you will have to > > recreate the filesystem. > > My Orangefs is newly created and the test program is the first one run in > parallel on it. > Isn't my configuration legit (one metadata sever and 4 data servers) for > orangefs setup? > > Again, my mpi test program ran 2 processes locally on the metadata server > which is both data server and client. > > Wei-keng > > > > > Becky > > > > On Mon, Mar 16, 2015 at 11:30 AM, Wei-keng Liao > > <[email protected]> wrote: > > HI, Becky > > > > I tried the command option "-a 0 -n 0" and restart the client/server, but > > the same issue persists. > > pvfs2-ping command shows one metadata server and 4 data servers. > > I ran my test program on the metadata server. > > > > meta servers: > > tcp://bigdata:3334 Ok > > > > data servers: > > tcp://bigdata:3334 Ok > > tcp://bigdata1:3334 Ok > > tcp://bigdata2:3334 Ok > > tcp://bigdata3:3334 Ok > > > > > > Wei-keng > > > > On Mar 16, 2015, at 8:26 AM, Becky Ligon wrote: > > > > > Caching is still an issue if you have servers on more than one machine > > > and those servers provide metadata. Even in a one-server environment, it > > > could make a difference. > > > > > > The "ls" command uses the kernel module and client core, which in turn > > > use the caches, while the pvfs2-ls command does not. > > > > > > If you don't have the appropriate sudo permissions to modify the /proc > > > filesystem, then you can start the client with the caches turned off. > > > > > > Example: > > > > > > ./pvfs2-client -a 0 -n 0 > > > > > > > > > If you execute pvfs2-client --help, you will see these options. > > > > > > > > > Becky > > > > > > Sent from my iPhone > > > > > >> On Mar 15, 2015, at 5:05 PM, Wei-keng Liao > > >> <[email protected]> wrote: > > >> > > >> I assume after 60 seconds, the client will flush the cache. > > >> Please note I am running orangefs client and server on the same machine. > > >> In this case, should caching become an issue? > > >> > > >> Long after 60 seconds of the file creation, command ls still could not > > >> find the file. > > >> > > >> I got permission denied when running the two echo commands you suggested. > > >> I DO have sudo permission. I also tried vi those files but got error of > > >> "/proc/sys/pvfs2/acache/timeout-msecs" E667: Fsync failed > > >> > > >> Also, how do I set this automatically after system reboot? > > >> > > >> Wei-keng > > >> > > >>> On Mar 15, 2015, at 2:16 PM, Becky Ligon wrote: > > >>> > > >>> Wei-keng: > > >>> > > >>> This is most likely a caching issue with the client. By default, we > > >>> set the cache to timeout after 60 seconds, which may be too high in > > >>> your environment. Or, you have deleted and redefined a file using the > > >>> same name outside of the client where you are seeing the question > > >>> marks, in which case, the cache would be wrong for that file. > > >>> > > >>> To verify, turn off caching to see if this resolves your problem: > > >>> > > >>> As root on your client machine: > > >>> > > >>> echo "0" > /proc/sys/pvfs2/acache/timeout-msecs > > >>> echo "0" > /proc/sys/pvfs2/ncache/timeout-msecs > > >>> > > >>> If this change fixes your problem, try setting the timeout-msecs to > > >>> something more appropriate for your environment. > > >>> > > >>> Becky > > >>> > > >>> On Sun, Mar 15, 2015 at 11:43 AM, Wei-keng Liao > > >>> <[email protected]> wrote: > > >>> Hi > > >>> > > >>> I am having problems with OrangeFS 2.9.1 and MPICH 3.1.4. > > >>> > > >>> Here is my system settings: > > >>> Linux Kernel 2.6.32 > > >>> Berkeley DB version 6.1.19 > > >>> > > >>> A simple MPI test program that calls MPI_File_open and > > >>> MPI_File_write_all > > >>> is used and ran two processes on the same host. > > >>> > > >>> The MPI commands I used with prefix file names to force ADIO drivers: > > >>> mpiexec -n 2 coll_write /orangefs/wkliao/testfile > > >>> mpiexec -n 2 coll_write pvfs2:/orangefs/wkliao/testfile.pvfs2 > > >>> mpiexec -n 2 coll_write ufs:/orangefs/wkliao/testfile.ufs > > >>> The first two will use the pvfs2 driver and the 3rd the ufs driver. > > >>> > > >>> Here is what I see when running "ls -l" and "pvfs2-ls -l" commands. > > >>> > > >>> % ls -l /orangefs/wkliao/ > > >>> ls: cannot access /orangefs/wkliao/testfile: No such file or directory > > >>> ls: cannot access /orangefs/wkliao/testfile.pvfs2: No such file or > > >>> directory > > >>> total 31252 > > >>> ?????????? ? ? ? ? ? testfile > > >>> ?????????? ? ? ? ? ? testfile.pvfs2 > > >>> -rw------- 1 wkliao users 32000000 Mar 13 18:55 testfile.ufs > > >>> > > >>> % pvfs2-ls -l /orangefs/wkliao/ > > >>> -rw-r--r-- 1 wkliao users 31000000 2015-03-13 18:55 testfile > > >>> -rw------- 1 wkliao users 32000000 2015-03-13 18:55 > > >>> testfile.ufs > > >>> -rw-r--r-- 1 wkliao users 31000000 2015-03-13 18:55 > > >>> testfile.pvfs2 > > >>> > > >>> My config.log file for building orangefs can be found in this URL > > >>> http://www.ece.northwestern.edu/~wkliao/config.log > > >>> > > >>> Wei-keng > > >>> > > >>> _______________________________________________ > > >>> Pvfs2-users mailing list > > >>> [email protected] > > >>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users > > >> > > > > > > > > > > -- > > Becky Ligon > > Research Associate > > Clemson University > > Clemson, SC > > > > > -- > Becky Ligon > Research Associate > Clemson University > Clemson, SC > > > > -- > Becky Ligon > Research Associate > Clemson University > Clemson, SC _______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
