Re: [Gluster-devel] [CentOS-devel] Multiple Geo Rep issues due to SELINUX on CentOS 8.3
Hi Srijan, I have compiled the rpm from the git repo and zeroed-out the lab and redeployed with that package: [root@glustera ~]# rpm -qa | grep glusterfs-selinux glusterfs-selinux-0.1.0-2.el8.noarch [root@glustera ~]# rpm -ql glusterfs-selinux /usr/share/selinux/devel/include/contrib/ipp-glusterd.if /usr/share/selinux/packages/targeted/glusterd.pp.bz2 /var/lib/selinux/targeted/active/modules/200/glusterd Yet , the result is very bad. I had to set SELINUX in permissive on all source nodes to establish a successfull geo-rep and all 7 denials have occured again on the source volume nodes: [root@glustera ~]# sealert -a /var/log/audit/audit.log 100% done found 7 alerts in /var/log/audit/audit.log - --- Are you sure that glusterfs-selinux is suitable for EL8 and glusterfs v8.3 ? Best Regards, Strahil Nikolov В 19:47 + на 06.01.2021 (ср), Strahil Nikolov via CentOS-devel написа: > Hi Srijan, > > I just checked the gluster repo 'centos-gluster8' and it seems that > there is no gluster package containing 'selinux' in the name . Same > is valid for CentOS 7. > Maybe I have to address that to the CentOS Storage SIG ? > > > Best Regards, > Strahil Nikolov > > > > > > > В сряда, 6 януари 2021 г., 10:44:37 Гринуич+2, Strahil Nikolov via > CentOS-devel написа: > > > > > > Hi Srijan, > > I will redeploy the scenario and I will check if the steps include > that package. > Shouldn't glusterfs-selinux a dependency ? > > Best Regards, > Strahil Nikolov > > > > > > > В сряда, 6 януари 2021 г., 07:29:27 Гринуич+2, Srijan Sivakumar < > ssiva...@redhat.com> написа: > > > > > > Hi Strahil, > > Selinux policies and rules have to be added for gluster processes to > work as intended when selinux is in enforced mode. Could you confirm > if you've installed the glusterfs-selinux package in the nodes ? > If not then you can check out the repo at > https://github.com/gluster/glusterfs-selinux. > > Regards, > Srijan > > On Wed, Jan 6, 2021 at 2:15 AM Strahil Nikolov > wrote: > > Did anyone receive that e-mail ? > > Any hints ? > > > > Best Regards, > > Strahil Nikolov > > > > В 19:05 + на 30.12.2020 (ср), Strahil Nikolov написа: > > > Hello All, > > > > > > I have been testing Geo Replication on Gluster v 8.3 ontop CentOS > > > 8.3. > > > It seems that everything works untill SELINUX is added to the > > > equasion. > > > > > > So far I have identified several issues on the Master Volume's > > > nodes: > > > - /usr/lib/ld-linux-x86-64.so.2 has a different SELINUX Context > > > than > > > the target that it is pointing to. For details check > > > https://bugzilla.redhat.com/show_bug.cgi?id=1911133 > > > > > > - SELINUX prevents /usr/bin/ssh from search access to > > > /var/lib/glusterd/geo-replication/secret.pem > > > > > > - SELinux is preventing /usr/bin/ssh from search access to .ssh > > > > > > - SELinux is preventing /usr/bin/ssh from search access to > > > /tmp/gsyncd-aux-ssh- > > > tnwpw5tx/274d5d142b02f84644d658beaf86edae.sock > > > > > > Note: Using 'semanage fcontext' doesn't work due to the fact that > > > files created are inheriting the SELINUX context of the parent > > > dir > > > and you need to restorecon after every file creation by the geo- > > > replication process. > > > > > > - SELinux is preventing /usr/bin/rsync from search access on > > > .gfid/----0001 > > > > > > Obviously, those needs fixing before anyone is able to use Geo- > > > Replication with SELINUX enabled on the "master" volume nodes. > > > > > > Should I open a bugzilla at bugzilla.redhat.com for the selinux > > > policy? > > > > > > Further details: > > > [root@glustera ~]# cat /etc/centos-release > > > CentOS Linux release 8.3.2011 > > > > > > [root@glustera ~]# rpm -qa | grep selinux | sort > > > libselinux-2.9-4.el8_3.x86_64 > > > libselinux-utils-2.9-4.el8_3.x86_64 > > > python3-libselinux-2.9-4.el8_3.x86_64 > > > rpm-plugin-selinux-4.14.3-4.el8.x86_64 > > > selinux-policy-3.14.3-54.el8.noarch > > > selinux-policy-devel-3.14.3-54.el8.noarch > > > selinux-policy-doc-3.14.3-54.el8.noarch > > > selinux-policy-targeted-3.14.3-54.el8.noarch > > > > > > [root@glustera ~]# rpm -qa | grep gluster | sort > > > centos-release-gluster8-1.0-1.el8.noarch > > > glusterfs-8.3-1.el8.x86_64 > > > glusterfs-cli-8.3-1.el8.x86_64 > > > glusterfs-client-xlators-8.3-1.el8.x86_64 > > > glusterfs-fuse-8.3-1.el8.x86_64 > > > glusterfs-geo-replication-8.3-1.el8.x86_64 > > > glusterfs-server-8.3-1.el8.x86_64 > > > libglusterd0-8.3-1.el8.x86_64 > > > libglusterfs0-8.3-1.el8.x86_64 > > > python3-gluster-8.3-1.el8.x86_64 > > > > > > > > > [root@glustera ~]# gluster volume info primary > > > > > > Volume Name: primary > > > Type: Distributed-Replicate > > > Volume ID: 89903ca4-9817-4c6f-99de-5fb3e6fd10e7 > > > Status: Started > > > Snapshot Count: 0 > > > Number of Bricks: 5 x 3 = 15 > > >
Re: [Gluster-devel] [CentOS-devel] Multiple Geo Rep issues due to SELINUX on CentOS 8.3
Hi Srijan, I just checked the gluster repo 'centos-gluster8' and it seems that there is no gluster package containing 'selinux' in the name . Same is valid for CentOS 7. Maybe I have to address that to the CentOS Storage SIG ? Best Regards, Strahil Nikolov В сряда, 6 януари 2021 г., 10:44:37 Гринуич+2, Strahil Nikolov via CentOS-devel написа: Hi Srijan, I will redeploy the scenario and I will check if the steps include that package. Shouldn't glusterfs-selinux a dependency ? Best Regards, Strahil Nikolov В сряда, 6 януари 2021 г., 07:29:27 Гринуич+2, Srijan Sivakumar написа: Hi Strahil, Selinux policies and rules have to be added for gluster processes to work as intended when selinux is in enforced mode. Could you confirm if you've installed the glusterfs-selinux package in the nodes ? If not then you can check out the repo at https://github.com/gluster/glusterfs-selinux. Regards, Srijan On Wed, Jan 6, 2021 at 2:15 AM Strahil Nikolov wrote: > Did anyone receive that e-mail ? > Any hints ? > > Best Regards, > Strahil Nikolov > > В 19:05 + на 30.12.2020 (ср), Strahil Nikolov написа: >> Hello All, >> >> I have been testing Geo Replication on Gluster v 8.3 ontop CentOS >> 8.3. >> It seems that everything works untill SELINUX is added to the >> equasion. >> >> So far I have identified several issues on the Master Volume's nodes: >> - /usr/lib/ld-linux-x86-64.so.2 has a different SELINUX Context than >> the target that it is pointing to. For details check >> https://bugzilla.redhat.com/show_bug.cgi?id=1911133 >> >> - SELINUX prevents /usr/bin/ssh from search access to >> /var/lib/glusterd/geo-replication/secret.pem >> >> - SELinux is preventing /usr/bin/ssh from search access to .ssh >> >> - SELinux is preventing /usr/bin/ssh from search access to >> /tmp/gsyncd-aux-ssh-tnwpw5tx/274d5d142b02f84644d658beaf86edae.sock >> >> Note: Using 'semanage fcontext' doesn't work due to the fact that >> files created are inheriting the SELINUX context of the parent dir >> and you need to restorecon after every file creation by the geo- >> replication process. >> >> - SELinux is preventing /usr/bin/rsync from search access on >> .gfid/----0001 >> >> Obviously, those needs fixing before anyone is able to use Geo- >> Replication with SELINUX enabled on the "master" volume nodes. >> >> Should I open a bugzilla at bugzilla.redhat.com for the selinux >> policy? >> >> Further details: >> [root@glustera ~]# cat /etc/centos-release >> CentOS Linux release 8.3.2011 >> >> [root@glustera ~]# rpm -qa | grep selinux | sort >> libselinux-2.9-4.el8_3.x86_64 >> libselinux-utils-2.9-4.el8_3.x86_64 >> python3-libselinux-2.9-4.el8_3.x86_64 >> rpm-plugin-selinux-4.14.3-4.el8.x86_64 >> selinux-policy-3.14.3-54.el8.noarch >> selinux-policy-devel-3.14.3-54.el8.noarch >> selinux-policy-doc-3.14.3-54.el8.noarch >> selinux-policy-targeted-3.14.3-54.el8.noarch >> >> [root@glustera ~]# rpm -qa | grep gluster | sort >> centos-release-gluster8-1.0-1.el8.noarch >> glusterfs-8.3-1.el8.x86_64 >> glusterfs-cli-8.3-1.el8.x86_64 >> glusterfs-client-xlators-8.3-1.el8.x86_64 >> glusterfs-fuse-8.3-1.el8.x86_64 >> glusterfs-geo-replication-8.3-1.el8.x86_64 >> glusterfs-server-8.3-1.el8.x86_64 >> libglusterd0-8.3-1.el8.x86_64 >> libglusterfs0-8.3-1.el8.x86_64 >> python3-gluster-8.3-1.el8.x86_64 >> >> >> [root@glustera ~]# gluster volume info primary >> >> Volume Name: primary >> Type: Distributed-Replicate >> Volume ID: 89903ca4-9817-4c6f-99de-5fb3e6fd10e7 >> Status: Started >> Snapshot Count: 0 >> Number of Bricks: 5 x 3 = 15 >> Transport-type: tcp >> Bricks: >> Brick1: glustera:/bricks/brick-a1/brick >> Brick2: glusterb:/bricks/brick-b1/brick >> Brick3: glusterc:/bricks/brick-c1/brick >> Brick4: glustera:/bricks/brick-a2/brick >> Brick5: glusterb:/bricks/brick-b2/brick >> Brick6: glusterc:/bricks/brick-c2/brick >> Brick7: glustera:/bricks/brick-a3/brick >> Brick8: glusterb:/bricks/brick-b3/brick >> Brick9: glusterc:/bricks/brick-c3/brick >> Brick10: glustera:/bricks/brick-a4/brick >> Brick11: glusterb:/bricks/brick-b4/brick >> Brick12: glusterc:/bricks/brick-c4/brick >> Brick13: glustera:/bricks/brick-a5/brick >> Brick14: glusterb:/bricks/brick-b5/brick >> Brick15: glusterc:/bricks/brick-c5/brick >> Options Reconfigured: >> changelog.changelog: on >> geo-replication.ignore-pid-check: on >> geo-replication.indexing: on >> storage.fips-mode-rchecksum: on >> transport.address-family: inet >> nfs.disable: on >> performance.client-io-threads: off >> cluster.enable-shared-storage: enable >> >> I'm attaching the audit log and sealert analysis from glustera (one >> of the 3 nodes consisting of the 'master' volume). >> >> >> Best Regards, >> Strahil Nikolov > >> > > --- > > Community Meeting Calendar: > Schedule - > Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC > Bridge: https://meet.google.com/cpu-eiue-hvk > > Gluster-devel mailing list > Gluster-devel@gluster.org
