Thank you for carefully considering suggestions (and implications) - and which will 'work' for you.

Further comment below (and with apologies that, unusually for me, there are many personal opinions mixed-in):-


On 06/04/2023 01.06, Loris Bennett wrote:
"Loris Bennett" <loris.benn...@fu-berlin.de> writes:
dn <pythonl...@danceswithmice.info> writes:
On 01/04/2023 02.01, Loris Bennett wrote:
Hi,
In my top level program file, main.py, I have
    def main_function():
        parser = argparse.ArgumentParser(description="my prog")
        ...
        args = parser.parse_args()
        config = configparser.ConfigParser()
        if args.config_file is None:
            config_file = DEFAULT_CONFIG_FILE
        else:
            config_file = args.config_file
        config.read(config_file)
        logging.config.fileConfig(fname=config_file)
        logger = logging.getLogger(__name__)
        do_some_stuff()
               my_class_instance = myprog.MyClass()
    def do_some_stuff():
        logger.info("Doing stuff")
This does not work, because 'logger' is not known in the function
'do_some_stuff'.
However, if in 'my_prog/my_class.py' I have
    class MyClass:
        def __init__(self):
            logger.debug("created instance of MyClass")
this 'just works'.
I can add
    logger = logging.getLogger(__name__)
to 'do_some_stuff', but why is this necessary in this case but not
in
the class?
Or should I be doing this entirely differently?

Yes: differently.

To complement @Peter's response, two items for consideration:

1 once main_function() has completed, have it return logger and other
such values/constructs. The target-identifiers on the LHS of the
function-call will thus be within the global scope.

2 if the purposes of main_function() are condensed-down to a few
(English, or ..., language) phrases, the word "and" will feature, eg
- configure env according to cmdLN args,
- establish log(s),
- do_some_stuff(),  ** AND **
- instantiate MyClass.

If these (and do_some_stuff(), like MyClass' methods) were split into
separate functions* might you find it easier to see them as separate
sub-solutions? Each sub-solution would be able to contribute to the
whole - the earlier ones as creating (outputting) a description,
constraint, or basis; which becomes input to a later function/method.

So if I want to modify the logging via the command line I might have the
following:

---------------------------------------------------------------------

#!/usr/bin/env python3

import argparse
import logging


def get_logger(log_level):
     """Get global logger"""

     logger = logging.getLogger('example')
     logger.setLevel(log_level)
     ch = logging.StreamHandler()
     formatter = logging.Formatter('%(levelname)s - %(message)s')
     ch.setFormatter(formatter)
     logger.addHandler(ch)

     return logger


def do_stuff():
     """Do some stuff"""

#    logger.info("Doing stuff!")

Looks like I just need

   logger = logging.getLogger('example)
   logger.info("Doing stuff!")


def main():
     """Main"""

     parser = argparse.ArgumentParser()
     parser.add_argument("--log-level", dest="log_level", type=int)
     args = parser.parse_args()

     print(f"log level: {args.log_level}")

     logger = get_logger(args.log_level)
     logger.debug("Logger!")
     do_stuff()


if __name__ == "__main__":
     main()

---------------------------------------------------------------------

How can I get logging for 'do_stuff' in this case without explicitly
passing 'logger' as an argument or using 'global'?

Somehow I am failing to understand how to get 'logger' defined
sufficiently high up in the program that all references 'lower down' in
the program will be automatically resolved.

At the risk of 'heresy', IMHO the idea of main() is (almost always) unnecessary in Python, and largely a habit carried-over from other languages (need for an entry-/end-point).

NB be sure of the difference between a "script" and a "module"...


My script template-overview:

''' Script docstring. '''
- author, license, etc docs

global constants such as import-s
set environment

if __name__ == "__main__":
    do_this()
    do_that()
    ie the business of the script


Despite its frequent use, I'm somewhat amused by the apparent duplication within:

if __name__ == "__main__":
    main()

ie if this .py file is being executed as a script, call main() - where main() is the purpose of the script. Whereas if the file is an import-ed module, do not execute any of the contained-code.

Thus, the if-statement achieves the same separation as the main() function encapsulation. Also, the word main() conveys considerably less meaning than (say) establish_logging().


NB others may care to offer alternate advice for your consideration...


There is a good argument for main(), in that it might enable tests to be written which assure the integration of several tasks called by the script. Plus, if one excludes non-TDD test-able code, such as user interfaces, even 'fancy headings' and printing of conclusions; there's little left. Accordingly, I don't mind the apparent duplication involved in coding an integration test which checks that do_this() and do_that() are playing-nicely together. YMMV!


I'm not sure why, but my 'mainlines' never seem to be very long. Once I had been introduced to "Modular Programming" (1970s?), it seemed that the purpose of a mainline was merely calling one do-it function after another.

To me mainline's seem highly unlikely to be re-usable. Thus, the possibility that main() might need to be import-able to some other script, is beyond my experience/recollection.

Further, my .py file scripts/mainlines tend to be short and often don't contain any of the functions or classes which actually do-the-work - all are import-ed (and thus, they are easier to re-use!?)


Returning to 'set environment': the code-examples include argparse and logger. Both (in my thinking) are part of creating the environment in which the code will execute.

Logging for example, is the only choice when we need to be aware of how a web-based system is running (or running into problems) - otherwise, as some would argue, we might as well use print(). Similarly, argparse will often influence the way a script executes, the data it should use, etc.

Much of such forms the (global) environment in which (this run of) the code will execute. Hence locating such setting of priorities and constraints, adjacent to the import statements.

These must be established before getting on with 'the business'. That said, if someone prefers to put it under if __main__, I won't burst into tears. (see earlier comment about the .py file as a script cf module)


You have answered your own question about logging. Well done!

The logging instance can either be explicitly passed into each (relevant) function, or it can be treated as a global and thus available implicitly. (see also: "Zen of Python")

I prefer your approach of get_logger() - even if that name doesn't quite describe the establishment of a logger and its settings. All of that part of setting the environment is collected into one place. Which in-turn, makes it easy to work on any changes, or work-in any parameters which may come from other environment-setting activity.


As well as 'the documentation' there is a HowTo (https://docs.python.org/3/howto/logging.html). Other logging-learners have found the DigitalOcean tutorial helpful (https://www.digitalocean.com/community/tutorials/how-to-use-logging-in-python-3)


* there is some debate amongst developers about whether "one function,
   one purpose" should be a rule, a convention, or tossed in the
  trash. YMMV!

Personal view: SOLID's "Single" principle applies: there should be
only one reason (hanging over the head of each method/function, like
the Sword of Damocles) for it to change - or one 'user' who could
demand a change to that function. In other words, an updated cmdLN
option shouldn't affect a function which establishes logging, for
example.


Web.Refs:
https://people.engr.tamu.edu/choe/choe/courses/20fall/315/lectures/slide23-solid.pdf
https://www.hanselminutes.com/145/solid-principles-with-uncle-bob-robert-c-martin
https://idioms.thefreedictionary.com/sword+of+Damocles
https://en.wikipedia.org/wiki/Damocles

I don't really get the "one reason" idea and the Sword of Damocles
analogy.  The later to me is more like "there's always a downside",
since the perks of being king may mean someone might try to usurp the
throne and kill you.  Where is the "single principle" aspect?

However, the idea of "one responsibility" in the sense of "do only one
thing" seems relatively clear, especially if I think in terms of writing
unit tests.

+1

Users are notoriously unable to write clear specifications. Ultimately, and not necessarily unkindly, they often do not know what they want - and particularly not specified at the level of detail we need. This is the downfall of the "waterfall" SDLC development model, and the reason why short 'Agile' sprints are (often/in-theory) more successful. The sooner a mistake, misunderstanding, or omission is realised, the better!

Not wanting to do more than state the above, things change, life goes on, etc, etc. So, when one first demonstrates code to a user, it is rare that they won't want to change something. Similarly, over time, it is highly unlikely that someone won't dream up some 'improvement' - or similar need to amend the code be imposed by an externality, eg government legislation or regulation. Such changes may be trivial, eg cosmetic; others may be more far-reaching, eg imposition of a tax where previously there was none (or v-v).

Accordingly, adding the fourth dimension to one's code, and program[me]-design - and the advice about being kind to those who will maintain the code after you or your six-months-time-self.

So, the 'sword of Damocles' is knowing that our code should not be considered secure (or static). That change is inevitable. We don't need to be worrying about danger afflicting us when we least expect it - your users are unlikely to actually kill you. However, it is a good idea to be aware that change may be required, and that it could come from any random direction or "stakeholder".

Perhaps worse, change implies risk. We make a change to suit one aspect, and something else 'breaks'. That is the reason many programmers actively resist 'change'. That is also one of the promises of TDD - if we can quickly run tests which assure (if not "prove") code-correctness, then the risks of change decrease. Whereas, if there are no tests to ensure 'life goes on' after a change, well, unhappiness reigns!

Hence the allusion.
(apologies if it was too oblique)


Accordingly, the idea that if a function does one job (and does it well), should you decide/be required to change that function, then the impacts, any side-effects, etc, of such will only affect that one function (and its tests), and whatever is 'downstream' (integrating) from there.

Which also reduces the chance that some change 'over here', will have some unforeseen impact 'over there'. The structure and the flow work together to achieve both a degree of separation and a bringing-together or unity. The tension of team-work if you like, cf the dysfunction of disorganisation.

NB not that a 'small change' means running only that one test in TDD - still run the entire suite of tests (auto-magically) to ensure that there are no surprises...


Change is inevitable. Thus, consider that your throne as code-author is illusory - the user/client is able/likely to want change.

Then, if the code is organised according to functionality, the logging environment (for example) can be changed without any need to re-code somewhere else - and/or that tinkering with cmdLN arguments and such code-changes can be done quite separately from the establishment of logging.

Ultimately, the smaller the function (think of establishing logging), the more likely it will be able to be re-used in the next assignment which requires a log! (whereas if it is mixed-in with argparse, re-use is unlikely because the cmdLN args will be different)


There's plenty of references about such on the web. Will be happy to discuss whichever sub-topics might be of-interest, further...

--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

Reply via email to