Re: Friday Finking: 'main-lines' are best kept short
On Thu, Sep 12, 2019 at 10:59 PM DL Neil via Python-list wrote: > Ref: > Mastering Object-oriented Python, S Lott > Copyright © 2014 Packt Publishing Side note: When I looked this up I saw on Amazon that there is a second edition out targeting Python 3.7. It was published June of this year. -- boB -- https://mail.python.org/mailman/listinfo/python-list
Re: Friday Finking: 'main-lines' are best kept short
On 13Sep2019 15:58, DL Neil wrote: Is it a good idea to keep a system's main-line* code as short as possible, essentially consigning all of 'the action' to application and external packages and modules? Generally yes. * my choice of term: "main-line", may be taken to mean: - the contents of main(), - the 'then clause' of an if __name__ == __main__: construct, - a __main__.py script. Taking these out of order: I don't like "if __name__ == '__main__':" to be more than a few lines. If it gets past about 4 or 5 then I rip it out into a main() function and use: if __name__ == '__main__': sys.exit(main(sys.argv)) and put "def main(argv):" at the top of the module (where it is glaringly obvious). Once at that stage, where you have a __main__.py or a "def main()" is based _entirely_ on whether this is a module or a package. There is no other criterion for me. [... snip ...] Doesn't the author thus suggest that the script (main-line of the program) should be seen as non-importable? __main__.py is generally something you would never import, any more than you would want to import the _body_ of a main() function. Particularly because it will run things that have side effects; a normal import should not. Doesn't he also suggest that the script not contain anything that might be re-usable? That is a very similar statement, or at least tightly tied in. If you can't import __main__.py because it actually runs the main programme, then you can't import it to make use of resuable things. Therefore reusable things should not have their definitions in __main__.py. Accordingly, the script calls packages/modules which are both importable and re-usable. None of which discounts the possibility of having other 'main-lines' to execute sub-components of the (total) application, should that be appropriate. An issue with 'main-line' scripts is that they can become difficult to test - or to build, using TDD and pytest (speaking personally). Pytest is great for unit tests, and can be used for integration testing, but the 'higher up' the testing pyramid we go, the less effectual it becomes (please don't shoot me, pytest is still an indispensable tool!) Accordingly, if 'the action' is pushed up/out to modules, this will ease the testing, by access and by context! Yes. So ideally your "main" should be fairly skeletal, calling out to components defined elsewhere. To round things out, I seem to be structuring projects as: .projectV2 -- README -- LICENSE -- docs (sub-directory) -- .git (sub-directory) -- etc -- __main__.py [...] I don't have a top level __main__.py in the project source tree; I _hate_ having python scripts in the top level because they leak into the import namespace courtesy of Python's sys.path including the current directory. __main__.py belongs in the package, and that is down a level (or so) from the main source tree. [...] Part of making the top-level "projectV2" directory almost-irrelevant in day-to-day dev-work is that __main__.py contains very little, typically three stages: 1 config (including start logging, etc, as appropriate) 2 create the applications central/action object 3 terminate Nary an if __name__ == __main__ in sight (per my last "Wednesday Wondering"), because "the plan" says there is zero likelihood of the "main-line" being treated as a (re-usable) module! (and any refactoring would, in any case, involve pushing such code out to a (re-usable) module! As alluded to earlier, the "if __main__ == '__main__':" is entirely an idiom to support main-programme semantics in a module. In a package you have a __main__.py and no need for the idiom. When it comes to execution, the command (excluding any switches/options) becomes: [~/Projects]$ python3 projectV2 And there's your use case for the top level __main__.py. I prefer: python3 -m projectv2 where the projectv2 package is found via the sys.path. In production projectv2 would be installed somewhere sensible, and in development I've a little "dev" shell function which presumes it is in the project top level and sets $PATH, $PYTHPATH etc to allow "python3 -m projectv2" to find the package. So in dev I go: dev python3 -m projectv2 The advantage here is that if I don't prefix things with "dev" I get the official installed projectv2 (whatever that means - it couldeasily be my personal ~/bin etc), and with the "dev" prefix I get the version in my development tree. So that I don'trun the dev stuff by accident (which is one reason I eschew the virtualenv "activate" script - my command line environment should not be using the dev environment inless I say so, because "dev" might be broken^Wincompatible). Which would also distinguish between project-versions, if relevant. More importantly, changes to application version numbers do not require any changes to import statements! (and when users don't wish to be expected to remember
Friday Finking: 'main-lines' are best kept short
(this follows some feedback from the recent thread: "WedWonder: Scripts and Modules" and commences a somewhat-related topic/invitation to debate/correct/educate) Is it a good idea to keep a system's main-line* code as short as possible, essentially consigning all of 'the action' to application and external packages and modules? * my choice of term: "main-line", may be taken to mean: - the contents of main(), - the 'then clause' of an if __name__ == __main__: construct, - a __main__.py script. In a previous thread I related some ?good, old days stories. When we tried to break monolithic programs down into modular units, a 'rule of thumb' was "one page-length" per module (back in the mainframe days our code was 'displayed' on lineflo(w) (continuous stationery) which was 66 - call it 60, lines per page - and back-then we could force a page-break where it suited us!). Then when we moved to time-share screens (80 characters by 24 lines), we thought that a good module-length should conform to screen-size. These days I have a large screen mounted in 'portrait mode', so on that basis I'm probably back to 50~60 lines (yes, these old eyes prefer a larger font - cue yet more cheeky, age-ist comments coming from my colleagues...) Likely I have also picked-up and taken-to-heart the *nix mantra of code doing 'one job, and doing it well' (and hence the extensive powers of redirects, piping, etc - in Python we 'chain' code-units together with "import"). Accordingly, I tend to err on the side of short units of code, and thus more methods/functions than others might write. In "Mastering Object-oriented Python" the author discusses "Designing a main script and the __main__ module" (Ch17): <<< A top-level main script will execute our application. In some cases, we may have multiple main scripts because our application does several things. We have three general approaches to writing the top-level main script: • For very small applications, we can run the application with python3.3 some_script.py . This is the style that we've shown you in most examples. • For some larger applications, we'll have one or more files that we mark as executable with the OS chmod +x command. We can put these executable files into Python's scripts directory with our setup.py installation. We run these applications with some_script.py at the command line. • For complex applications, we might add a __main__.py module in the application's package. To provide a tidy interface, the standard library offers the runpy module and the -m command-line option that will use this specially named module. We can run this with python3.3 -m some_app. [explanation of "shebang" line - the second approach, above] Creating a __main__ module To work with the runpy interface, we have a simple implementation. We add a small __main__.py module to our application's top-level package. We have emphasized the design of this top-level executable script file. We should always permit refactoring an application to build a larger, more sophisticated composite application. If there's functionality buried in __main__.py , we need to pull this into a module with a clear, importable name so that it can be used by other applications. A __main__.py module should be something small like the following code: import simulation with simulation.Logging_Config(): with simulation.Application_Config() as config: main= simulation.Simulate_Command() main.config= config main.run() We've done the minimum to create the working contexts for our application. All of the real processing is imported from the package. Also, we've assumed that this __main__.py module will never be imported. This is about all that should be in a __main__ module. Our goal is to maximize the reuse potential of our application. [example] We shouldn't need to create composite Python applications via the command-line API. In order to create a sensible composition of the existing applications, we might be forced to refactor stats/__main__.py to remove any definitions from this module and push them up into the package as a whole. >>> Doesn't the author thus suggest that the script (main-line of the program) should be seen as non-importable? Doesn't he also suggest that the script not contain anything that might be re-usable? Accordingly, the script calls packages/modules which are both importable and re-usable. None of which discounts the possibility of having other 'main-lines' to execute sub-components of the (total) application, should that be appropriate. An issue with 'main-line' scripts is that they can become difficult to test - or to build, using TDD and pytest (speaking personally). Pytest is great for unit tests, and can be used for integration testing, but the 'higher up' the testing pyramid we go, the less effectual it becomes (please don't
