On Jun 24, 2014, at 10:16 AM, Jan Lieskovsky <[email protected]> wrote:
> ----- Original Message ----- >> From: "Steve Grubb" <[email protected]> >> To: [email protected] >> Cc: "Jan Lieskovsky" <[email protected]> >> Sent: Tuesday, June 24, 2014 2:32:43 PM >> Subject: Re: Formalizing the release process / schedule >> >> On Tuesday, June 24, 2014 04:55:33 AM Jan Lieskovsky wrote: >>> Hello folks, >>> >>> since the request to give more formalized shape to SSG release process / >>> schedule appeared couple of times in the past already (to mention some >>> example: >>> https://lists.fedorahosted.org/pipermail/scap-security-guide/2014-May/00541 >>> 6.html) >> >> I think a faster release cadence is a good idea in general. But beside >> speeding it up, some more formalities need to be considered. The biggest one >> is what is the meaning of major, minor, and patch in the project versioning? >> When would there be a stable branch? > > Thank you, Steve. These are great questions. To be honest I am not able to > answer > them, but will try (Shawn, Jeff please correct me where appropriate): > * patch version - no new features, just bug fixes for existing checks > * minor version - adding new XCCDF rules / OVAL checks > * major version - adding support for new product? > > In the current scenario we don't differentiate these from each other (test > case > fix might go into release with new rule definition / new product support). > After > some time (when master diverged "sufficiently enough" from existing released > version) > a new tarball is released. The current approach results into situation we are > not > able (at least I) to tell how is e.g. 0.1.18 version better than 0.1.17 one > was. > So at least from this point of view having the versioning rules more strictly > defined > it might be easier for the consumers to know what got improved from the last > version. > > Regarding the stable branch - from the nowadays experience "it seems" the > patches > are merged to stable branch in the moment they have shown not to break things. > What I personally miss from the version scheme being some way of expression > about > the percentage, how much of the system is covered by the rules and to how > much level > of testing those rules have been actually tested. In other words so the > consumer > would be able to say e.g. v0.1.17 has had the RHEL-6 product covered to 75% > percentage, > with 57% of rules being covered with test_attestation. v0.1.18 should then > improve > these statistics (at least I hope so) - not just for RHEL-6, but also for > other products. > > Having this applied I think it would be easier to identify the state, when we > have reached > the "stable status" for some product. We could define the percentage level > (for both cases) > -- border of percentage that once reached the particular product content > should be > declared as stable. Then there are tricky things like how to express > particular benchmark's > section is complete? Should various sections have various contribution to the > final result / > stamp for content for particular product? > > Maybe they are other ways how to measure stability of a content for > particular product > (if you have proposals for other different metrics feel free to share them). > > To implement these metrics it might take some additional time / effort, but > at the end it > could allow us to support expressions about stability of the product with > scientific methods. > >> >> Then, based on having a versioning policy, its desirable to overlay that with >> goals. What are the goals for 0.2? 0.3? 1.0? > > Here again. These are white places on the map for now. Not able to tell how > would > v0.3 be different from v0.2 -- is it just v0.3 would have more rules > implemented, more > tests equipped with attestations, more products supported? Or would we want > to differentiate > also based on the features? If so, what features would be the key ones the > presence of them > by themselves to be sufficient enough for claiming new major version? > >> >> As the project grows in importance, these need to be documented so that >> consumers know what to expect. > > Agree on the need of these to be explicitly expressed (if not now yet, they > will definitely be needed with more urgency in the future). > > Thank you && Regards, Jan. > -- > Jan iankko Lieskovsky / Red Hat Security Technologies Team > >> >> -Steve >> >> >>> and since we are hitting the doubt if already make / not to make yet a new >>> release each time (each couple of weeks), we have decided to share the idea >>> about SSG release process proposal. >>> >>> The proposal is to have release of new tarballs to happen at regular time >>> (each 4 up to 6 weeks), and have a dedicated wiki page listing when the >>> couple of upcoming releases will happen [1]. Something like Mozilla is >>> doing for their products: https://wiki.mozilla.org/Releases >>> https://wiki.mozilla.org/Releases#Upcoming_Releases >>> https://wiki.mozilla.org/RapidRelease/Calendar >>> >>> From the observation period of 6 weeks is enough to have "sufficient >>> count >>> of new bug fixes / features requests" (aka the evolution step) these >>> changes by themselves to form a need of a new release. But since it's >>> difficult to decide if some patchset deserves new release (what's very >>> valuable for someone, might not be that exciting / urgent for someone else >>> etc.) to be fair we decided regular release period might be the best option >>> how to balance out the various requests that might come out. >>> >>> Therefore starting from the next release we would like the release to >>> happen regularly (each 4 up to 6 weeks - this to be decided yet. Feel free >>> to vote for your preferences) at a date listed on the wiki page [2]. This >>> could allow us establish automated functionality which could in >>> sufficiently ahead period of time (for example week ago) notify the list >>> the new release is coming and if someone is planning from their point of >>> view critical patches to be included yet, those should be proposed / >>> reviewed as soon as possible. >>> >>> To express personal opinion I would prefer the releases to happen on per >>> 4 >>> week scenario (while 6 weeks window might produce more changes in the >>> tarball & less releases during the year at the end it also introduces some >>> related inconsistency - one time the release would happen at start of the >>> month, next time in the middle etc.) since it has the advantage the release >>> will happen each time at regular (same) period, so users can update their >>> functionality to better align upgrades with the schedule. >>> >>> Yet, since majority of us works on various / multiple projects, it might >>> happen start of the week might not be the right time for new release >>> (urgent action required somewhere else resulting into delay of a release >>> etc.). Therefore I would propose the release to happen each first Friday in >>> the month (the users could use the upcoming weekend to install the updates >>> if / where appropriate). >>> >>> Feedback, comments, votes, proposals welcome. >>> >>> Thank you && Regards, Jan. >>> -- >>> Jan iankko Lieskovsky / Red Hat Security Technologies Team (on behalf of >>> SSG >>> upstream) >>> >>> [1] For now the wiki would contain just dates of past releases & dates of >>> the upcoming ones (possibly also describing the release scheme). In the >>> future it could provide further information what functionality each of the >>> releases introduced (and taking it to the very advanced level hopefully too >>> which functionality the new releases will contain in the future). >>> >>> [2] Once the scheme is clear (the proposal is approved), I will create that >>> wiki page. _______________________________________________ >>> scap-security-guide mailing list >>> [email protected] >>> https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide >> >> > _______________________________________________ > scap-security-guide mailing list > [email protected] > https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide I agree that a time based release makes the most sense. I additionally think that a monthly release, either the first Friday or last Friday makes a lot of sense. Presumably a release on a Friday would mean a new package by the following Friday. 2 weeks in testing for EPEL (until shipped with RHEL) would mean the package would be ready for general consumption 3-4 weeks later. Bleeding edge consumers could help test the package and more risk adverse consumers could continue testing and just remain a month or more behind in releases. As far as versioning is concerned, I believe there should be more discussion around it and what the different versions mean. I don’t think this should interfere with the decision on a time-based release schedule. Just my 0.02. -josh _______________________________________________ scap-security-guide mailing list [email protected] https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide
