Hi, Here are the various artifacts from this meeting,
1) Recording: https://drive.google.com/file/d/0B_1LWY0UTlNiQTI0ZU9hWDJGdEk/view?usp=sharing
2) Slides: https://docs.google.com/presentation/d/1SNzak0lyEaKh2Zin-P1v9JUD_uSodiIVAssKOfJj7Ow/edit?usp=sharing
3) Attached please find steps to replay the demo, building from sources. (Nov_5_2015_StatusUpdateDemo.log)
Thank you, Shyam/Venky On 11/03/2015 09:27 PM, Shyam wrote:
Hi, Coming Thursday, i.e Nov-05-2015, @ 6:30 AM Eastern we are having a short call to present DHT2 status (ics attached) To join the meeting on a computer or mobile phone: https://bluejeans.com/371209168?src=calendarLink&g=onzgc3thmfxgcqdsmvsgqylufzrw63i= Agenda: - Current status update - Quick demo of some FOPs working (lookup, create) - A quick tour on how things look at the backend - Targets for next milestone We will record the meeting, so in case you are interested but are unable to make it, a recording will be posted. We will also post a mail update on the status, so finer details should be present in text form anyway. Thanks, Shyam _______________________________________________ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
How/Where to get the code ========================= 1) git: git clone ssh://shyamsund...@review.gerrithub.io:29418/ShyamsundarR/glusterfs 2) cd to cloned directory and cherry pick the following changes that are not yet merged, - https://review.gerrithub.io/#/c/251309/ (FYI, the above brings in 2 other changes when cherry picked) - https://review.gerrithub.io/#/c/250059/ - https://review.gerrithub.io/#/c/251134/ - https://review.gerrithub.io/#/c/251377/ - https://review.gerrithub.io/#/c/250778/ NOTE: Post the above cherry picks the git log should look as below (changes may have moved, so commit # may not match), # git log -n8 --decorate --pretty=oneline --abbrev-commit d485bdc (HEAD -> integration) dht2/glusterd: Added graph generation change to include posix2-mds 20e7dab Integration: Fixes for remote inode lookup to work ad4255b storage/posix2: Inode operations 4ba5572 posix, v2: Boilerplate for mds, ds split 2e6e550 (review/shyam/dht2fops-30) cluster/dht2: Add prototype open/stat/ setattr/flush FOPs bee2a3e (review/shyam/dht2fops-20) cluster/dht2: Prototype implementation of create aa21512 (review/shyam/dht2fops-10) cluster/dht2: Prototype implementation of lookup FOP c67933b (origin/gl40_dht_playground, gerrit/gl40_dht_playground, gl40_dht_playground) dht2: Initial conceptual design for the second DHT2 prototype 3) Build (./autogen.sh; ./configure; make clean; make -j8) -> Install (sudo make install) 4) Get to a root shell and play with the steps below... GOTCHA'S: - Create "/d/backends/" directory before proceeding with the steps. - GFID generation would differ as the steps are run, as they are randomly generated - Mode bits may differ as per umask of systems used to run the commands - Do not use tab completions (or ls like commands) for paths on the gluster mount, readdir(p) is not implemented yet (the brick process will crash) Step 1: Gluster health checks ============================= # glusterd # gluster v status No volumes present Step 2: Creating a DHT2 volume ============================== # gluster volume create patchy mds 2 data 1 \ marvin.bos.redhat.com:/d/backends/patchy1 \ marvin.bos.redhat.com:/d/backends/patchy2 \ marvin.bos.redhat.com:/d/backends/patchy3 # gluster v status Volume patchy is not started Inspecting the vol files for DHT2 and POSIX2 changes ---------------------------------------------------- # less /var/lib/glusterd/vols/patchy/trusted-patchy.tcp-fuse.vol (Note distribute2/dht2 instead of regular dht in the vol file) # less \ /var/lib/glusterd/vols/patchy/\ patchy.marvin.bos.redhat.com.d-backends-patchy1.vol (Note posix2-mds instead of regular posix in the vol file) Inspecting the bricks pre-volume start -------------------------------------- # tree /d/backends/patchy? /d/backends/patchy1 /d/backends/patchy2 /d/backends/patchy3 0 directories, 0 files Starting the volume and inspecting the bricks --------------------------------------------- # gluster v start patchy volume start: patchy: success # gluster v status Status of volume: patchy Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------ Brick marvin.bos.redhat.com:/d/backends/pat chy1 49152 0 Y 21075 Brick marvin.bos.redhat.com:/d/backends/pat chy2 49153 0 Y 21090 Brick marvin.bos.redhat.com:/d/backends/pat chy3 49154 0 Y 21105 NFS Server on localhost N/A N/A N N/A Task Status of Volume patchy ------------------------------------------------------------------------------ There are no active volume tasks # tree /d/backends/patchy? /d/backends/patchy1 └── 00 └── 00 └── 00000000-0000-0000-0000-000000000001 /d/backends/patchy2 /d/backends/patchy3 3 directories, 0 files (Note ['root'|'/'] GFID 0x00...01 created by default) Step 3: Mounting and inspecting attributes of gluster root ========================================================== # mount -t glusterfs marvin.bos.redhat.com:/patchy /mnt/glusterfs/0 # stat -c %A /mnt/glusterfs/0 drwx------ # stat -c %A /d/backends/patchy1/00/00/00000000-0000-0000-0000-000000000001/ drwx------ Changing the mode for root on the brick and checking back on the client ----------------------------------------------------------------------- # chmod 770 /d/backends/patchy1/00/00/00000000-0000-0000-0000-000000000001/ # stat -c %A /mnt/glusterfs/0 drwxrwx--- Step 4: Creating a file and inspecting the bricks ================================================= # touch /mnt/glusterfs/0/File1 # tree /d/backends/patchy? /d/backends/patchy1 └── 00 └── 00 ├── 00000000-0000-0000-0000-000000000001 │ └── File1 └── 00003e56-1b94-4f99-ac20-dab615aa06e8 /d/backends/patchy2 /d/backends/patchy3 3 directories, 2 files Demonstrating the relation between the file name and its inode/GFID ------------------------------------------------------------------- # getfattr -e hex -n trusted.gfid \ /d/backends/patchy1/00/00/00000000-0000-0000-0000-000000000001/File1 trusted.gfid=0x00003e561b944f99ac20dab615aa06e8 (Note File1 trusted.gfid xattr points to an on disk file of the same hexadecimal name, on the same brick (this is the DHT2 co-location policy, inode for files in a directory belong to the same brick)) Displaying the real meta-data store ----------------------------------- # stat -c %A \ /d/backends/patchy1/00/00/00000000-0000-0000-0000-000000000001/File1 -rwx------ # stat -c %A /d/backends/patchy1/00/00/00003e56-1b94-4f99-ac20-dab615aa06e8 -rw-r--r-- # stat -c %A /mnt/glusterfs/0/File1 -rw-r--r-- (Note that at the mount point the metadata for the object (a file here) is displayed based on its GFID object in the backend and not its dentry/name object) Step 5: chown (or setattr) functionality ======================================== # stat -c %U /mnt/glusterfs/0/File1 root # chown somari /mnt/glusterfs/0/File1 # stat -c %U /mnt/glusterfs/0/File1 somari # stat -c %U \ /d/backends/patchy1/00/00/00000000-0000-0000-0000-000000000001/File1 root # stat /d/backends/patchy1/00/00/00003e56-1b94-4f99-ac20-dab615aa06e8 somari (This is just to demonstrate that setattr works, and that the GFID is the one that is modified, as that rather than the dentry/name is the real metadata store) Step 6: Directory distibution ============================= NOTE: (synthetic creation as mkdir FOP is TBD) ---------------------------------------------- # tree /d/backends/patchy? /d/backends/patchy1 └── 00 └── 00 ├── 00000000-0000-0000-0000-000000000001 │ └── File1 └── 00003e56-1b94-4f99-ac20-dab615aa06e8 /d/backends/patchy2 /d/backends/patchy3 3 directories, 2 files # touch /d/backends/patchy1/00/00/00000000-0000-0000-0000-000000000001/Dir1 # mkdir -p /d/backends/patchy2/00/01/00010000-dddd-0000-0000-000000000001 # setfattr -n trusted.gfid -v 0x00010000dddd00000000000000000001 \ /d/backends/patchy1/00/00/00000000-0000-0000-0000-000000000001/Dir1 (Note, use the same GFID in both 'mkdir -p' and 'setfattr' and further keep the first 4 bytes the same, so that it falls logically into the second brick as per the client DHT2 layout) # tree /d/backends/patchy? /d/backends/patchy1 └── 00 └── 00 ├── 00000000-0000-0000-0000-000000000001 │ ├── Dir1 │ └── File1 └── 00003e56-1b94-4f99-ac20-dab615aa06e8 /d/backends/patchy2 └── 00 └── 01 └── 00010000-dddd-0000-0000-000000000001 /d/backends/patchy3 6 directories, 3 files (Note, the above demonstrates lack of co-location for directory objects, resulting in providing metadata distribution) Inspecting the directory from the client ---------------------------------------- # stat -c %A /mnt/glusterfs/0/Dir1 drwxr-xr-x # chmod 700 /d/backends/patchy2/00/01/00010000-dddd-0000-0000-000000000001/ # stat -c %A /mnt/glusterfs/0/Dir1 drwx------ Step 7: Creating files within sub-directories and inspecting the bricks ======================================================================= # touch /mnt/glusterfs/0/Dir1/File2 # tree /d/backends/patchy? /d/backends/patchy1 └── 00 └── 00 ├── 00000000-0000-0000-0000-000000000001 │ ├── Dir1 │ └── File1 └── 00003e56-1b94-4f99-ac20-dab615aa06e8 /d/backends/patchy2 └── 00 └── 01 ├── 00010000-dddd-0000-0000-000000000001 │ └── File2 └── 000175d5-2bc4-4f07-8bbb-27b42f30eaf4 /d/backends/patchy3 6 directories, 5 files # getfattr -n trusted.gfid -e hex \ /d/backends/patchy2/00/01/00010000-dddd-0000-0000-000000000001/File2 trusted.gfid=0x000175d52bc44f078bbb27b42f30eaf4 (Note similar dentry/name and GFID relations and co-location property as demonstrated for files within root, hold true for files within other subdirectories, but do not need to hold for directories) Step 8: Bonus demonstration of non co-located files (say due to rename) ======================================================================= (NOTE: Synthetic splitting of file name and inode done as rename FOP is TBD) ---------------------------------------------------------------------------- # touch /d/backends/patchy2/00/01/00010000-dddd-0000-0000-000000000001/File3 # setfattr -n trusted.gfid -v 0x00000000ffff00000000000000000001 \ /d/backends/patchy2/00/01/00010000-dddd-0000-0000-000000000001/File3 # touch /d/backends/patchy1/00/00/00000000-ffff-0000-0000-000000000001 (Note, the above simulates a condition where File3 was originally created under (say) root, and hence got a GFID heaader of 0x0000 to co-locate its inode on the same MDS. Further to this, say this file was renamed under Dir1, so here name moves, but inode/GFID does not change, causing a non co-located file name with its inode) # tree /d/backends/patchy? /d/backends/patchy1 └── 00 └── 00 ├── 00000000-0000-0000-0000-000000000001 │ ├── Dir1 │ └── File1 ├── 00000000-ffff-0000-0000-000000000001 └── 00003e56-1b94-4f99-ac20-dab615aa06e8 /d/backends/patchy2 └── 00 └── 01 ├── 00010000-dddd-0000-0000-000000000001 │ ├── File2 │ └── File3 └── 000175d5-2bc4-4f07-8bbb-27b42f30eaf4 /d/backends/patchy3 6 directories, 7 files # getfattr -n trusted.gfid -e hex \ /d/backends/patchy2/00/01/00010000-dddd-0000-0000-000000000001/File3 trusted.gfid=0x00000000ffff00000000000000000001 # stat -c %A /mnt/glusterfs/0/Dir1/File3 -rw-r--r-- [root@marvin glusterfs]# stat -c %A /mnt/glusterfs/0/Dir1/File3 -rw-r--r-- [root@marvin glusterfs]# chmod 777 \ /d/backends/patchy1/00/00/00000000-ffff-0000-0000-000000000001 [root@marvin glusterfs]# stat -c %A /mnt/glusterfs/0/Dir1/File3 -rwxrwxrwx (Note, the above inspects the metadata for the file, and also demonstrates changed inode (not entry/name metadata) information being reflected on the mount) <END>
_______________________________________________ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel