Hi, just a few thoughts about this. Generally - I like the idea of breaking this up.
I would like to list a few thoughts about additional technical points that we should perhaps think about and that play into this decision. * Testing: Currently, some of the policy scripts have tests that use Zeek functionality in rather unique ways / or are the only tests for some Zeek functionality. The SSL validation scripts are one example. This, from my point of view, it would be neat to have a way to still easily install a rather large set of packages (potentially nearly everything that is in policy at the moment) and run test on them. This also comes with a fun problem. Sometimes we perform changes to Zeek that change a lot of the test baselines - especially when we touch something that affects connection-ID hashing, or the order of elements in hashmaps. These cases might now require an update to the test-cases in a large number of packages. It would be neat to have an easy way to perform this. * Versioning: I think if we want to do this, we need to have a better story for versioning than we, at the moment, have with zkg. To expand on this - at the moment the policy scripts just work with the version of Zeek that we distribute them with. It would be nice if, afterwards, it would still be possible to install a working set of a script for the running version of Zeek. Meaning that - if someone happens to run a version of Zeek that is 12 months out of date - they should probably get the version of the policy script that is known to work with this version of Zeek and where the tests pass with this version of Zeek. It would be super nice if this worked rather fine-granular - so even for development versions of Zeek. * Documentation: At the moment the documentation of the policy scripts just lives together with the Zeek documentation. This has a few advantages - it e.g. shows redefs that are performed in policy scripts. It would be neat to have a place that contains the combined documentation of these scripts. Currently, especially given the "update-tons-of-test-baselines-simultaneously" problem, I am kind of tempted by 3. 3 would also enable relatively easy versioning and mapping to Zeek versions that the packages are known to work with. This should also allow to keep the current testing infrastructure more or less working as it is. It does, however, not give really fine-grained access to individual packages. Johanna On Mon, Aug 24, 2020 at 01:43:21PM +0000, Robin Sommer wrote: > Looking for some thoughts here. One of the items on the roadmap for > 4.0 is moving scripts that currently live in policy/ over into Zeek > packages. The goals here are to (1) facilitate maintaining & testing > them independently of Zeek releases; and (2) come to a more flexible > notion of "default scripts" that can incorporate community-maintained > packages as well. This is tracked by issue > https://github.com/zeek/zeek/issues/414, including a 1st pass over the > existing policy scripts to understand what should/can be moved. > (Thanks, Vlad!) > > Before we can begin working on this, we need to figure out how to > organize this new world. One particular question is where the moved > packages will live. I see the following options so far: > > 1. Move each into a a separate repository on the zeek/ GitHub > account. > > 2. Similar, but to avoid cluttering zeek/, create a new GitHub > organization "zeek-packages". > > 3. Put them all into a single mono-repository (e.g., > zeek/standard-packages), i.e., treat them a one package. > > 4. Do (1) or (2), and additionally create "zeek-standard-packages" > that's full of submodules pointing to them (and also to > community packages). > > 5. Do (1) or (2), and teach zkg to understand "collections" of > packages that can be installed/managed as a group, defined > through some meta data somewhere. > > Along with all of this comes a question of how to make it easy for > people to install a set of default packages now that these won't come > with Zeek itself anymore. Some of the schemes above make that easier > than others. > > Thoughts/opinions/more ideas? > > Robin > > -- > Robin Sommer * Corelight, Inc. * ro...@corelight.com * www.corelight.com > _______________________________________________ > zeek-dev mailing list -- zeek-dev@lists.zeek.org > To unsubscribe send an email to zeek-dev-le...@lists.zeek.org _______________________________________________ zeek-dev mailing list -- zeek-dev@lists.zeek.org To unsubscribe send an email to zeek-dev-le...@lists.zeek.org
