On 20/01/2020 22:37, Chris Johns wrote: > On 14/1/20 7:18 pm, Christian Mauderer wrote: >> On 13/01/2020 18:03, Joel Sherrill wrote: >>> >>> >>> On Mon, Jan 13, 2020 at 10:04 AM Christian Mauderer >>> <christian.maude...@embedded-brains.de >>> <mailto:christian.maude...@embedded-brains.de>> wrote: >>> >>> Hello Jose, >>> >>> On 10/01/2020 17:33, Jose Valdez wrote: >>> > Hello, >>> > >>> > Regarding this topic and to start to define the python tools that >>> are more appropriate for the RTEMS community python developments, I >>> would like to propose the following tools (to be placed in RTEMS >>> Software Engineering manual): >>> > >>> > Python language version: 3.6 (minimum version for Doorstop 2.0.post2) >>> > Python coding style/standard - Google (as suggested by Gedare) >>> > Python documentation style - Google docstrings (to be in >>> accordance with coding style. Note it is supported by sphinx) >>> > Python test approach - unittest (seems to be the standard used by >>> python) >>> > Python static analysis - pylint (recommended by Google style, see >>> http://google.github.io/styleguide/pyguide.html) and mypy (catch >>> additional errors) Python code coverage - Coverage.py (seems to be >>> the standard used by python) >>> > Python third party packages - For now we are using: paramiko, >>> pyserial, pexpect, gitpython, but the list is expected to be >>> changed. I think here any third package is allowed, as long as the >>> license of the package complies with RTEMS project license. >>> >>> To be future-proof: python3 compatibility would be nearly a must-have >>> for any new library. >>> >>> >>> We discussed this somewhere earlier and I thought we decided that if >>> it was a core tool required for a "normal user", then it would have to >>> be Python 2.x and 3.x compatible. Tools for "requirements user" or >>> "pre-qualification user" (pick a name for this role), can be 3.x only >>> mainly because there was so much infrastructure required for the >>> "requirements user" that already needed Python 3.x. > > Yes this is correct. Python is used in the rtems-tools repo and any program in > the rtems-tools repo needs to support Python 2 and Python 3 no matter what > role > it performs. The rtems-tools repo is not the same as doorstop or similar tools > used to develop RTEMS, they are user tools used to develop application to run > on > RTEMS.
Does that mean you don't want to have any pre-qualification tools in the rtems-tools repo? If yes: Where else? I fully agree that every tool that is thought for every day use for 99% of the RTEMS users should have Python 2 and 3 support. With no official Python 2 support since this year I don't think that this would be a good requirement for new specialized areas like pre-qualification. I think you agree there? On the other hand I wouldn't know a better place then rtems-tools for everything produced through the pre-qualification effort. So I'm not a fan of saying: Everything in rtems-tools should be Python 2 + 3. > > It is not clear to me what role the original python specification Jose listed > is > for. I will restrict my comments to rtems-tools or more generally any tool > installed into the user's tools prefix for use when developing RTEMS > applications. > > For a user focused tool: > > - Python is 2 and 3 > - No external third party packages, all support needs to be internal. What about stuff like pygdb (which is already used in rtems-tools)? > - Fully portable for all supported host platforms, that is Windows, MacOS, > FreeBSD and Linux. > > The second item is important. If user installed tools start to depend on 3rd > party packages the deployment complexity for us rises and the process becomes > complex. > > While on the topic of 3rd party packages I do not agree with Jose's view any > package can be used if the license it suitable. Package complexity, it's > dependencies and portability need to be considered, for example gitpython uses > gitdb and this is python code to directly read and write a git repo. I would > not > like any of our repo's being modified by that code so I would prefer we do not > use it. If we need to extract data I prefer the solution in the rtems-tools > toolkit so we should also consider using existing support first. OK. So rule for third party is: For everything that is used by 99% of our users: Basically don't use any third party. Exceptions for that user group have to be discussed (and are most likely rejected) on the mailing list. For special use cases: - Should be avoided if a build-in can be used instead. - Only if there isn't another package for the same purpose already in use. - Open source license that is not putting restrictions on our code (like a code generator that would put the generated code under GPL). - Portable to all mayor host systems (for RTEMS that's Windows, Linux, FreeBSD, MacOS). - Libraries that touch critical infrastructure would have to discussed thoroughly with the community before it can be added. About right? Best regards Christian > >> >> Yes, sorry. I used the wrong wording. What I meant was that we should >> avoid libraries that support only python 2. But you have added a lot of >> relevant details to this. >> >>> >>> I think this discussion should also include "documentation producer" >>> because that requires Python 3.x as well. >>> >>> We just need to be conscious of the user roles that the tools are >>> supporting and justify the minimum. This gives us at least 3 categories >>> of tools with potentially different minimum hosting requirements. For >>> example, we may not care if a certain set of tools runs on Mingw. >>> >>> Obviously documented somewhere. >>> >>> >>> >>> I don't think that there are heavy license problems. The python packages >>> are most likely only used in tools. So as long as they are open enough >>> to use them on all host platforms for all development situations that >>> shouldn't be a problem. Exception: If the package is used to generate >>> code with a specific license. That generated code would have to be >>> compatible with the RTEMS license. >>> >>> >>> +1 and very much non-viral and no advertising for generated code. >>> >>> >>> > >>> > Please tell me your opinions for each python tool and then we can >>> start to agree on what to use. Note that once this is defined, our >>> Qualification software will comply with the RTEMS community chosen >>> tools. >>> >>> If no one objects I would suggest to start documenting them in the RTEMS >>> Software Engineering manual. A small example project in the rtems-tools >>> repository that uses the suggested code checkers (meaning: calls them >>> during a waf build or a special 'waf check' target) would be great too. >>> That could be used for all future python based tools that are added to >>> rtems-tools (which includes the whole qualification tools). >>> >>> >>> Yes. It must be enforced. Whether it is a special step or not is up for >>> discussion. >>> My first thought is that a special step is easy to skip. >>> >>> Also can "check" be done as a pre-commit hook? >> >> Possible. But I'm not sure. We might can try it as soon as we have it >> running in an example. > > The problem I see with git hooks is the catch is too late. A patch is tested > then reviewed only for a push to fail and the review process needs to start > again or be restarted. > >>> And when the style is agreed to, should existing code be auto reformatted >>> to meet it? >> >> I think this could be migrated on a per-tool basis. If we have one >> project that is a sample for all future tools the checkers can be back >> ported to existing tools when there is time or a good opportunity. > > You need to convince me a Python reformatting tool does not break existing > python code. A misaligned indent changes the logic. > > Chris > -- -------------------------------------------- embedded brains GmbH Herr Christian Mauderer Dornierstr. 4 D-82178 Puchheim Germany email: christian.maude...@embedded-brains.de Phone: +49-89-18 94 741 - 18 Fax: +49-89-18 94 741 - 08 PGP: Public key available on request. Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG. _______________________________________________ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel