Hey Trey, IMO rather than read all files from Dir A and then reading all files from Dir B and then processing which content to filter, it would be better to list the content from the current working directory (CWD), check if the file is valid, a symlink (just in case there is need for one), or missing. If the file is a symlink or missing, get the actual file from the shared/oval directory. The logic would seem easier to implement that way.
Gabe On Thu, Apr 9, 2015 at 10:27 AM, Trey Henefield < [email protected]> wrote: > Hi Gabe, > > > > I probably should have been more specific as to the Makefile change, but > essentially, it just involved changing the combinechecks.py line from this: > > > > $(SHARED)/$(TRANS)/combinechecks.py $(CONF) $(IN)/checks > > $(OUT)/unlinked-$(PROD)-oval.xml > > > > To this: > > > > $(SHARED)/$(TRANS)/combinechecks.py $(CONF) $(IN)/checks > $(SHARED)/oval > $(OUT)/unlinked-$(PROD)-oval.xml > > > > That part seems to work great with the code I provided. But the filtering > of shared checks included is where the challenge lies. > > > > Best regards, > > > > Trey Henefield, CISSP > > Senior IAVA Engineer > > > > Ultra Electronics > > Advanced Tactical Systems, Inc. > > 4101 Smith School Road > > Building IV, Suite 100 > > Austin, TX 78744 USA > > > > [email protected] > > Tel: +1 512 327 6795 ext. 647 > > Fax: +1 512 327 8043 > > Mobile: +1 512 541 6450 > > > > www.ultra-ats.com > > > > *From:* [email protected] [mailto: > [email protected]] *On Behalf Of *Gabe > Alford > *Sent:* Thursday, April 09, 2015 9:59 AM > > *To:* SCAP Security Guide > *Subject:* Re: Shared Checks/Fixes ... > > > > On Thu, Apr 9, 2015 at 8:04 AM, Trey Henefield < > [email protected]> wrote: > > > > Thank you everyone for the responses! > > I completely agree with the benefits of the shared directory. That really > wasn't so much my concern, but rather a better way of utilizing it. > > As Jan pointed out, I am using a mixed Windows/Linux environment, making > it challenging to maintain the symbolic links when traversing different > filesystems (NTFS -> EXT). > > So far, I have looked at some minor adjustments that can be made to > shared/transforms/combinechecks.py and shared/transforms/combinefixes.py. > > I have tried adding an additional parameter through the Makefile to point > to the $(SHARED)/oval folder and adding the following to > shared/transforms/combinechecks.py: > > for filename in os.listdir(sys.argv[2]): > if filename.endswith(".xml"): > with open(sys.argv[2] + "/" + filename, 'r') as xml_file: > body = body + xml_file.read() > + > + for filename in os.listdir(sys.argv[3]): > + if filename.endswith(".xml"): > + with open(sys.argv[3] + "/" + filename, 'r') as xml_file: > + body = body + xml_file.read() > > > > You could do it this way for combinechecks.py: > > + SHARED = os.environ.get('SHARED_DIR') > > > for filename in os.listdir(sys.argv[2]): > if filename.endswith(".xml"): > > + local_path = sys.argv[2] + "/" + filename > > + if not os.path.islink(local_path): > > + xml_filename = local_path > > + else: > > + xml_filename = SHARED + "/oval/" + filename > - with open(sys.argv[2] + "/" + filename, 'r') as xml_file: > + with open(xml_filename, 'r') as xml_file: > > > > Then add the following to the Makefile(s): > > export SHARED_DIR=$(SHARED) > > > > This has the proper effect of adding all project-specific checks first, > then adding in all shared checks. If there is a shared check with the same > name as a project-specific check, the shared check will be excluded in > favor of the project-specific check. Although, I personally think the > identifier of each shared check should be prefaced with 'shared_' to help > identify it as a shared check. This could help to prevent duplicate IDs in > any case. > > The downside of the above code though is that it adds all oval checks in > the shared folder to the oval output of a specific project. It builds fine. > But 'make validate' complains about oval checks not being included in the > XCCDF profile. There should be a better way to filter xml file content > added from the shared folder to just those specific to the project's > supported platform. > > As Shawn has hinted, this could perhaps be resolved in some XSLT magic. > Are there any XSLT magicians out there? ;) This is a language I have yet to > conquer. > > Best regards, > > > Trey Henefield, CISSP > Senior IAVA Engineer > > Ultra Electronics > Advanced Tactical Systems, Inc. > 4101 Smith School Road > Building IV, Suite 100 > Austin, TX 78744 USA > > [email protected] > Tel: +1 512 327 6795 ext. 647 > Fax: +1 512 327 8043 > Mobile: +1 512 541 6450 > > www.ultra-ats.com > > -----Original Message----- > From: [email protected] [mailto: > [email protected]] On Behalf Of Jan > Lieskovsky > Sent: Thursday, April 09, 2015 7:31 AM > To: SCAP Security Guide > Subject: Re: Shared Checks/Fixes ... > > Hello Trey, > > ----- Original Message ----- > > From: "Trey Henefield" <[email protected]> > > To: "[email protected]" > > <[email protected]> > > Sent: Wednesday, April 8, 2015 11:13:48 PM > > Subject: Shared Checks/Fixes ... > > > > Greetings, > > > > > > > > I wanted to propose a change to the current structure in place for > > shared checks (shared/oval) and fixes (shared/fixes/bash). I was > > curious to get everyone’s opinion before committing. > > Thank you for opening this topic. > > > > > > > > > So the problem I see is with the symbolic linking of checks and fixes > > to the shared folder. I have found it problematic when working between > > different operating systems, file systems, and version control systems. > > Right from the beginning we knew the concept of symbolic links will > introduce issues when dealing with SSG content across multiple OS versions. > But so far there hasn't been urgent need to fix this (to be read as so far > there haven't been much reports current behaviour causing issues for > developers). > But as the count of the SCAP content products raises it's time to fix this. > > > > > > > > > Rather than creating symbolic links to certain oval checks in the > > shared/oval folder, we could choose to just process all oval checks in > > both the project’s checks folder and the shared/oval folder. > > The original idea behind introducing /shared directory was the following - > when trying to identify corresponding OVAL check for particular XCCDF rule > the following scheme should be applied: > * when there's is OVAL check for that product present in input/checks > directory for that product, this OVAL check should be used with priority, > * when there isn't, use the corresponding (equally named) OVAL check from > shared/ directory. > > During the time we advanced the idea / concept even more in the sense that > shared/ version of OVAL checks doesn't necessarily need to be working for > all of the products having these checks (that's the case below mentioning > there are shared/ OVAL checks used just for RHEL-7 & Fedora systems, while > RHEL-6 version is different / RHEL-6 product specific). > > > > > > > > > However, not all checks in the current ‘shared/oval’ folder are shared > > by all OS. For example, there are some that only apply to RHEL7 and > > Fedora, and not RHEL6. > > See the explanation above why that's the case. > > > > > > > > > Any thoughts? > > If I should attempt to summary: > * the concept of shared/ checks has proven to be beneficial (in the sense > it's > less work to keep just one concrete copy updated than try to keep updated > multiple > identical copies), > * the mechanism of symbolic links is not the way to go (since it's known > not to work > e.g. on Microsoft Windows OS systems), > > So IMHO we want to keep the shared/ OVAL checks / fixes concept, but > implement it different way. Maybe as already suggested by modificating the > build process to look not just into CWD for OVAL check / remediation fix, > but also to look into particular subdirectory of shared/ directory (OVAL > checks vs fixes) simultaneously with preserving priority (if there's OVAL > check / fix present for that product, use that one. If there isn't and > there's is symlink, use the /shared version. If there isn't even symlink, > return no check / no fix available leading to 'notchecked' state). > > HTH > > Regards, Jan. > -- > Jan iankko Lieskovsky / Red Hat Security Technologies Team > > P.S.: As the count of products we produce SCAP content for is only going to > raise in the future, IMHO concept of /shared directory for OVAL checks & > fixes can only save us a lot of duplicate work in the future (when compared > to the scenario of managing multiple copies of the same file on per product > basis). > > > > > > > > > Thanks! > > > > > > > > Best regards, > > > > > > > > > > > > Trey Henefield, CISSP > > > > Senior IAVA Engineer > > > > > > > > Ultra Electronics > > > > Advanced Tactical Systems, Inc. > > > > 4101 Smith School Road > > > > Building IV, Suite 100 > > > > Austin, TX 78744 USA > > > > > > > > [email protected] > > > > Tel: +1 512 327 6795 ext. 647 > > > > Fax: +1 512 327 8043 > > > > Mobile: +1 512 541 6450 > > > > > > > > www.ultra-ats.com > > > > > > > > > > > > > > Disclaimer > > The information contained in this communication from > > [email protected] sent at 2015-04-08 17:13:55 is private > > and may be legally privileged or export controlled. It is intended > > solely for use by [email protected] and > > others authorized to receive it. If you are not > > [email protected] you are hereby notified > > that any disclosure, copying, distribution or taking action in > > reliance of the contents of this information is strictly prohibited and > may be unlawful. > > > > > > -- > > SCAP Security Guide mailing list > > [email protected] > > https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide > > https://github.com/OpenSCAP/scap-security-guide/ > -- > SCAP Security Guide mailing list > [email protected] > https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide > https://github.com/OpenSCAP/scap-security-guide/ > > *Disclaimer* > The information contained in this communication from > *[email protected] > <[email protected]> *sent at 2015-04-09 10:04:14 is private > and may be legally privileged or export controlled. It is intended solely > for use by *[email protected] > <[email protected]> *and others authorized to > receive it. If you are not *[email protected] > <[email protected]> *you are hereby notified > that any disclosure, copying, distribution or taking action in reliance of > the contents of this information is strictly prohibited and may be unlawful. > > > -- > SCAP Security Guide mailing list > [email protected] > https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide > https://github.com/OpenSCAP/scap-security-guide/ > > > > -- > SCAP Security Guide mailing list > [email protected] > https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide > https://github.com/OpenSCAP/scap-security-guide/ >
-- SCAP Security Guide mailing list [email protected] https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide https://github.com/OpenSCAP/scap-security-guide/
