That did not fix the issue (see below). I also have run into another
possibly related issue. After untarring the boost directory and
compiling the software, I cannot delete the source directory structure.
It says "direct not empty".
corvidpost5:temp3/gfs> \rm -r boost_1_57_0
rm: cannot remove `boost_1_57_0/libs/numeric/odeint/test': Directory not
empty
corvidpost5:temp3/gfs> cd boost_1_57_0/libs/numeric/odeint/test/
corvidpost5:odeint/test> ls -al
total 0
drwxr-x--- 3 dfrobins users 94 Dec 20 01:51 ./
drwx------ 3 dfrobins users 100 Dec 20 01:51 ../
cluster.read-hash-mode to 2 results
corvidpost5:TankExamples/DakotaList> ls -al
total 5
drwxr-x--- 2 dfrobins users 166 Dec 22 11:16 ./
drwxr-x--- 6 dfrobins users 445 Dec 22 11:16 ../
lrwxrwxrwx 1 dfrobins users 25 Dec 22 11:16 EvalTank.py ->
../tank_model/EvalTank.py*
---------- 1 dfrobins users 0 Dec 22 11:16 FEMTank.py
-rwx--x--- 1 dfrobins users 734 Nov 7 11:05 RunTank.sh*
-rw------- 1 dfrobins users 1432 Nov 7 11:05 dakota_PandL_list.in
-rw------- 1 dfrobins users 1860 Nov 7 11:05 dakota_Ponly_list.in
gluster> volume info homegfs
Volume Name: homegfs
Type: Distributed-Replicate
Volume ID: 1e32672a-f1b7-4b58-ba94-58c085e59071
Status: Started
Number of Bricks: 4 x 2 = 8
Transport-type: tcp
Bricks:
Brick1: gfsib01a.corvidtec.com:/data/brick01a/homegfs
Brick2: gfsib01b.corvidtec.com:/data/brick01b/homegfs
Brick3: gfsib01a.corvidtec.com:/data/brick02a/homegfs
Brick4: gfsib01b.corvidtec.com:/data/brick02b/homegfs
Brick5: gfsib02a.corvidtec.com:/data/brick01a/homegfs
Brick6: gfsib02b.corvidtec.com:/data/brick01b/homegfs
Brick7: gfsib02a.corvidtec.com:/data/brick02a/homegfs
Brick8: gfsib02b.corvidtec.com:/data/brick02b/homegfs
Options Reconfigured:
cluster.read-hash-mode: 2
performance.stat-prefetch: off
performance.io-thread-count: 32
performance.cache-size: 128MB
performance.write-behind-window-size: 128MB
server.allow-insecure: on
network.ping-timeout: 10
storage.owner-gid: 100
geo-replication.indexing: off
geo-replication.ignore-pid-check: on
changelog.changelog: on
changelog.fsync-interval: 3
changelog.rollover-time: 15
server.manage-gids: on
------ Original Message ------
From: "Vijay Bellur" <vbel...@redhat.com>
To: "David F. Robinson" <david.robin...@corvidtec.com>
Cc: "Justin Clift" <jus...@gluster.org>; "Gluster Devel"
<gluster-devel@gluster.org>
Sent: 12/22/2014 9:23:44 AM
Subject: Re: [Gluster-devel] 3.6.1 issue
On 12/21/2014 11:10 PM, David F. Robinson wrote:
So for now it is up to all of the individual users to know they cannot
use tar without the -P switch if they are accessing a data storage
system that uses gluster?
Setting volume option cluster.read-hash-mode to 2 could help here. Can
you please check if this resolves the problem without -P switch?
-Vijay
On Dec 21, 2014, at 12:30 PM, Vijay Bellur <vbel...@redhat.com>
wrote:
On 12/20/2014 12:09 PM, David F. Robinson wrote:
Seems to work with "-xPf". I obviously couldn't check all of the
files,
but the two specific ones that I noted in my original email do not
show
any problems when using -P...
This is related to the way tar extracts symbolic links by default &
its interaction with GlusterFS. In a nutshell the following steps are
involved in creation of symbolic links on the destination:
a) Create an empty regular placeholder file with permission bits set
to 0 and the name being that of the symlink source file.
b) Record the device, inode numbers and the mtime of the placeholder
file through stat.
c) After the first pass of extraction is complete, there is a second
pass involved to set right symbolic links. In this phase a stat is
performed on the placeholder file. If all attributes recorded in b)
are in sync with the latest information from stat buf, only then the
placeholder is unlinked and a new symbolic link is created. If any
attribute is out of sync, the unlink and creation of symbolic link do
not happen.
In the case of replicated GlusterFS volumes, the mtimes can vary
across nodes during the creation of placeholder files. If the stat
calls in steps b) and c) land on different nodes, then there is a
very good likelihood that tar would skip creation of symbolic links
and leave behind the placeholder files.
A little more detail about this particular implementation behavior of
symlinks for tar can be found at [1].
To overcome this behavior, we can make use of the P switch with tar
command during extraction which will create the link file directly
and not go ahead with the above set of steps.
Keeping timestamps in sync across the cluster will help to an extent
in preventing this situation. There are ongoing refinements in
replicate's selection of read-child which will help in addressing
this problem.
-Vijay
[1] http://lists.debian.org/debian-user/2003/03/msg03249.html
_______________________________________________
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel
