On Wed, May 8, 2019 at 8:37 AM Pierre Labastie <[email protected]> wrote: > > Hi, > > After some discussion in an pull request at github [1], it's been decided to > move the discussion to this list. > > Let's first (attempt to) summarize [1]: Jeremy attempted to write a Python > wrapper to the current jhalfs script, which was working well, but differently > from the usual jhalfs. So it was first decided to change the code in order to > have the same behavior as before. That turned out to be difficult without > adding some ad hoc code, which eventually would have become needlessly > complicated. It was then decided to start from scratch. I then asked for some > planning ahead. Jeremy then closed the pull request, and we decided to move > the planning to this list. > > I'd like to propose here a structure for the internal data of the jhalfs > program. This is open to discussion, of course. In the following, I propose > objects and attributes to those objects. I do not want to tell how those > attributes are assigned to, nor how they are used. Just have a data structure > universal enough so that different types of books can be fit in. Note that the > following is informal: the aim is to fix where the information should be > stored, not how (dictionary, list, string, number, ...). > > The top object could be 'book', with the following attributes: > - url (where to find the source) > - branch-or-tag (when using code repositories) > - language (for example 'docbook-xml', 'other-xml', 'markdown', 'rst', ...) > - name (for example 'lfs', 'clfs', 'diylinux', 'blfs', ...) > - need-user (a boolean indicating whether creating a user is needed for > running the book) > - if need-user: > - username > - group name > - environment > - parameters: for example how to replace <replaceable> tags for 'docbook-xml'. > Also whether to run the tests, to strip binaries, to remove .la files. Maybe > break this into several attributes, or have an object for that, or do that at > the package level (see below) > - dependency-type: 'ordered' (for example for lfs), 'package' (means each > package has its own deps), ...) > - parts: list of 'part' objects, which are described below > > The object 'part' may not be a reflect of the parts in the source book: the > idea is to break the book into parts that are run in the same environment > (chroot, run as user, run as root, run after reboot). Note that those parts > may also be used to sort the packages (such as for blfs, where everything is > run in the same environment, but parts are needed to make groups of chapters. > Otherwise, you'll have to choose among 800+ packages in 40+ chapters). That > is, several parts may have the same environment. Here are the attributes: > - title > - how-to-run: 'chroot', 'run-as-user', 'run-as-root', 'run-after-reboot', > 'mixed' (means "build as user and install as root") > - chapters: list of 'chapter' objects, which are described below > > The object 'chapter' is intermediate between part and page. It's main use is > to group similar packages (and/or config instructions). Proposed attributes: > - title > - pages: list of 'page' objects, which are described below > > The object 'page' sometimes has several packages. For example glibc in lfs has > glibc and tzdata, and several pages in blfs have 2 or more packages (perl and > python modules, Xorg libraries, fonts, and apps, kde frameworks, plasma, and > pages with more or less related packages such as poppler). Here are the > proposed attributes: > - title > - type ('single-package', 'multi-package', 'config-only') > - if single-package: > - package (see 'package' object description below) > - if multi-package: > - packages: list of 'package' objects > - if config-only: > - status ('installed', 'not-installed', needed since there is no > versioning) > - instructions: list of 'instruction' objects, to be described below > > The object 'package' attributes are: > - id (unique name) > - version > - url (list, to be tried in order) > - tarball name > - unpack-method ('tar', 'bsdtar', 'unzip', 'other') > - dependencies: list of couples ('package' objects <-> type), where type is > 'before', 'after', 'first' (in case there is a circular dep: a package with > this type should be built first with '-pass1' added to name, then rebuilt > after) > - status: 'installed, 'not-installed', 'old-version' > - needed: one or more of the following: 'basic', 'tests', 'build-doc', > 'time-size' (for timing and size measurements) > - instructions: ordered list of 'instruction' objects, to be described below > > The object 'instruction' has the following attributes: > - type: 'config-pre' (e.g.: user creation, directory settings), 'config-post' > (e.g.: update-icon-cache, various set-up), 'build' (may be refine into > 'configure', 'make', etc), 'install', 'test', 'build-doc' > - text (last but not least...) > > We may add a higher object: collection, pointing to a group of books, for > example lfs+blfs, to be able to build lfs+enough of blfs to run jhalfs... > > Hope this can make a starting point. Once we agree on this, we'll need to > write methods to convert some book (maybe a sample book for tests) to internal > data, and methods to convert our internal data to (scripts+program to run > them). The program to run the scripts can be a Makefile or something else: > we'll have to decide.
This generally sounds like a good approach. I hesitate to call out a specific design outside of the code, mostly because I believe the code should document well the design and it should also allow for some flexibility as we learn best how to organize and test. But I do think organizing by objects is a good way to look at it and I understand you are trying to avoid unbounded sprawl that becomes unmaintainable. So I think in that spirit it would be good to highlight just a few more things. 1. Have some kind of goal or mission statement for the code. Is the purpose to automate just xLFS style books, or is it a more generic bootstrapping tool? What would constitute out-of-scope? Try to keep it high level, but specific enough so that when new ideas or suggestions arise, we can firmly decide if it matches defined scope for the tool. 2. Let testability drive the shape of the code itself. Sometimes you have a structure well in mind for how the code should look and it makes a lot of sense, very organized and logical. But something specific about it makes it hard to test. Since testing drives quality, relaxing the design a little as you learn how best to test the code can make a huge difference. 3. Try to include in the code the structure required for good documentation, both when reading the code directly and possibly for auto-generating html docs, like with sphinx: http://www.sphinx-doc.org/en/stable/ JH -- http://lists.linuxfromscratch.org/listinfo/alfs-discuss FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
