Re: venv --upgrade 3.12.0rc2 --> 3.12.0rc3 failure
On 28/09/2023 10:05, Barry via Python-list wrote: So this must be the source of my confusion user@host:~ $ python312 -mvenv --help .. --upgrade Upgrade the environment directory to use this version of Python, assuming Python has been upgraded in-place. .. I have a different version, but it's not 'in place'. thanks -- Robin On 27 Sep 2023, at 12:50, Robin Becker via Python-list wrote: Attempting venv upgrade 3.12.0rc2 --> 3.12.0rc3 I find pyvenv.cfg changes, but the virtual python doesn't. I guess this ought to be a bug. You must delete and then recreate the venv if the version of python changes. It is not a bug in python. Barry -- https://mail.python.org/mailman/listinfo/python-list
venv --upgrade 3.12.0rc2 --> 3.12.0rc3 failure
Attempting venv upgrade 3.12.0rc2 --> 3.12.0rc3 I find pyvenv.cfg changes, but the virtual python doesn't. I guess this ought to be a bug. user@host:~/devel $ ~/LOCAL?3.12.0rc2/bin/python3 -m venv xxx bash: /home/user/LOCAL?3.12.0rc2/bin/python3: No such file or directory user@host:~/devel $ ~/LOCAL/3.12.0rc2/bin/python3 -m venv xxx user@host:~/devel $ xxx/bin/python -c'import sys;print(sys.version)' 3.12.0rc2 (main, Sep 9 2023, 17:53:34) [GCC 13.2.1 20230801] user@host:~/devel $ cat xxx/pyvenv.cfg home = /home/user/LOCAL/3.12.0rc2/bin include-system-site-packages = false version = 3.12.0 executable = /home/user/LOCAL/3.12.0rc2/bin/python3.12 command = /home/user/LOCAL/3.12.0rc2/bin/python3 -m venv /home/robin/devel/xxx user@host:~/devel $ ~/LOCAL/3.12.0rc3/bin/python3 -m venv --upgrade xxx user@host:~/devel $ xxx/bin/python -c'import sys;print(sys.version)' 3.12.0rc2 (main, Sep 9 2023, 17:53:34) [GCC 13.2.1 20230801] user@host:~/devel $ cat xxx/pyvenv.cfg home = /home/user/LOCAL/3.12.0rc3/bin include-system-site-packages = false version = 3.12.0 executable = /home/user/LOCAL/3.12.0rc3/bin/python3.12 command = /home/user/LOCAL/3.12.0rc3/bin/python3 -m venv --upgrade /home/robin/devel/xxx user@host:~/devel $ # check versions > user@host:~/devel $ ~/LOCAL/3.12.0rc2/bin/python3 -c'import sys;print(sys.version)' 3.12.0rc2 (main, Sep 9 2023, 17:53:34) [GCC 13.2.1 20230801] > user@host:~/devel $ ~/LOCAL/3.12.0rc3/bin/python3 -c'import sys;print(sys.version)' 3.12.0rc3 (main, Sep 27 2023, 09:35:10) [GCC 13.2.1 20230801] user@host:~/devel $ -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: [RELEASE] Python 3.12.0 beta 1 released.
On 25/05/2023 12:23, Robin Becker wrote: On 22/05/2023 22:04, Thomas Wouters wrote: > I'm pleased to announce the release of Python 3.12 beta 1 (and feature > freeze for Python 3.12). > ... I see a major difference between 3.12.0a7 and 3.12.0b1 Basically in preppy an importer is defined to handle imports of '.prep' files. This worked as expected in the a7 version and fails in the b1. I put in some prints in the code and I see these calls for the a7 run> $ ~/LOCAL/3.12.0a7/bin/python3 test_import.py > sys.meta_path.insert(.PreppyImporter object at 0x7fa870b84080>) --> . I think this might be caused by the removal of the find_module method of importlib.abc.MetaPathFinder. So I guess I have to implement a modernised importer. Apologies for noise if that is the case. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: [RELEASE] Python 3.12.0 beta 1 released.
On 22/05/2023 22:04, Thomas Wouters wrote:
> I'm pleased to announce the release of Python 3.12 beta 1 (and feature
> freeze for Python 3.12).
>
...
I see a major difference between 3.12.0a7 and 3.12.0b1
Basically in preppy an importer is defined to handle imports of '.prep' files.
This worked as expected in the a7 version and fails in the b1. I put in some prints in the code and I see these calls
for the a7 run> $ ~/LOCAL/3.12.0a7/bin/python3 test_import.py
> sys.meta_path.insert(.PreppyImporter object at 0x7fa870b84080>) -->
[.PreppyImporter object at 0x7fa870b84080>, <_distutils_hack.DistutilsMetaFinder object
at 0x7fa871290fb0>, , , '_frozen_importlib_external.PathFinder'>]
> PreppyImporter.find_module('sample001',None)
> PreppyImporter.load_module('sample001')
> 4
> .
> --
> Ran 1 test in 0.004s
>
> OK
In 3.12.0b1 although the importer is inserted into sys.meta_path the
find_module/load_module methods are never called.
and the import fails.
So is this an expected change in the way importers behave or a bug?
> $ ~/LOCAL/3.12.0b1/bin/python3 test_import.py
> sys.meta_path.insert(.PreppyImporter object at 0x7fc866ecb110>) -->
[.PreppyImporter object at 0x7fc866ecb110>, ,
, ]
> E
> ==
> ERROR: testImport1 (__main__.ImportTestCase.testImport1)
> --
> Traceback (most recent call last):
> File "/home/robin/devel/reportlab/REPOS/preppy/tmp/test_import.py", line
13, in testImport1
> import sample001
> ModuleNotFoundError: No module named 'sample001'
>
> --
> Ran 1 test in 0.001s
>
> FAILED (errors=1)
..
> Your release team,
> Thomas Wouters
> Ned Deily
> Steve Dower
--
https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.12.0 alpha 6 released
On 08/03/2023 04:37, Thomas Wouters wrote:
I'm pleased to announce the release of Python 3.12 alpha 6.
https://www.python.org/downloads/release/python-3120a6/
*This is an early developer preview of Python 3.12.*
Major new features of the 3.12 series, compared to 3.11
I was able to test reportlab with the 3.12.0a5 release (I build following the Archlinux pkgbuild), but am unable to do
so with 3.12.0a6 because of problems with cython/lxml/freetype-py (I think).
With an optimized build of a6 I was getting segfaults which I think were caused by incompatible C extensions from the
pip cache so decided to rebuild the requirements.
With latest cython git (allegedly 3.0.0b1) I see errors related to tstate eg
Cython/Compiler/Parsing.c:86861:34: error: ‘PyThreadState’ {aka ‘struct _ts’}
has no member named ‘curexc_traceback’
86861 | PyObject* tmp_tb = tstate->curexc_traceback;
I guess that this is caused by changes in what we are allowed to see of Python
internal structure.
If anyone knows of a way to advance further I can try to experiment.
Python 3.12 is still in development. This release, 3.12.0a6 is the sixth of
seven planned alpha releases.
Alpha releases are intended to make it easier to test the current state of
new features and bug fixes and to test the release process.
During the alpha phase, features may be added up until the start of the
beta phase (2023-05-08) and, if necessary, may be modified or deleted up
until the release candidate phase (2023-07-31). Please keep in mind that
this is a preview release and its use is not recommended for production
environments.
Many new features for Python 3.12 are still being planned and written.
Among the new major new features and changes so far:
Your release team,
Thomas Wouters
Ned Deily
Steve Dower
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: lxml with python-3.12.0a5
On 23/02/2023 18:09, Mats Wichmann wrote: I seem to always have trouble with lxml (which I know doesn't help). The cause would seem to be this: GH-101291: Refactor the `PyLongObject` struct into object header and PyLongValue struct. (GH-101292) So it looks to me like cython was affected, and has patched themselves. It's possible regenerating with a patched cython will clear up the build problem (something which the lxml project takes pains to tell you that you don't really want to do :) Thanks Mats, I build a cython wheel from git latest and after installing into python-3.12.0a5 I used latest lxml source and python setup.py bdist_wheel --with-cython which built without error. The installed lxml seems fine (at least for reportlab tests). -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
lxml with python-3.12.0a5
I'm trying to test python-3.12.0a5 and need to install lxml.
My wheel build for lxml fails with errors like this
src/lxml/etree.c: In function ‘__Pyx_PyIndex_AsSsize_t’:
src/lxml/etree.c:270404:45: error: ‘PyLongObject’ {aka ‘struct _longobject’}
has no member named ‘ob_digit’
270404 | const digit* digits = ((PyLongObject*)b)->ob_digit;
| ^~
I'm using archlinux which has gcc 12.2.1. I tested and lxml will build with
3.11.2.
I imagine this is part of ongoing changes to the python core, but perhaps
there's some simple workaround.
Anyone know how to get around this problem. I did try rebuilding the cpython
stuff using make, but that also failed.
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: C extension custom types in abi3 module
On 08/12/2022 12:52, Robin Becker wrote: I am trying to split off reportlab C extensions to simplify installations and make use of more advanced packages. A simple extension is easily converted to being an abi3 module. However, another has a custom type which uses the old style mechanisms here it looks like I have to use a different mechanism to setup custom types in the abi3 world. In the docs I see this "Also, since PyTypeObject is only part of the Limited API as an opaque struct, any extension modules using static types must be compiled for a specific Python minor version." So it seems I must switch to using a heap allocated type or keep compiling in the old way. I looked in Modules/xxlimited_35.c, but that seems much more complex and provides for a type which supports GC. Are there any ABI3 examples using the old style strategy? -- Robin Becker -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
C extension custom types in abi3 module
I am trying to split off reportlab C extensions to simplify installations and
make use of more advanced packages.
A simple extension is easily converted to being an abi3 module. However, another has a custom type which uses the old
style mechanisms here
https://docs.python.org/3.7/extending/newtypes_tutorial.html#the-basics
I made a simple setup based on this abi3 example modified to allow switching
between abi3 and normal build
https://github.com/joerick/python-abi3-package-sample
I made a tiny change to the code example in the newtypes tutorial page the code
is here
https://github.com/MrBitBucket/custom-type
In a python 3.7 - 3.12a3 environment I find I can build the wheel OK with
ABI3_WHEEL=0 pip wheel -w dist .
but I get lots of compile errors if I switch to an abi3 build with
ABI3_WHEEL=1 pip wheel -w dist .
looking at the errors
src/_custom.c:10:1: error: variable ‘CustomType’ has initializer but
incomplete type
10 | static PyTypeObject CustomType = {
| ^~
In file included from
/home/robin/LOCAL/3.7.16/include/python3.7m/Python.h:90,
from src/_custom.c:2:
/home/robin/LOCAL/3.7.16/include/python3.7m/object.h:90:5: error: extra
brace group at end of initializer
90 | { PyObject_HEAD_INIT(type) size },
| ^
src/_custom.c:11:9: note: in expansion of macro ‘PyVarObject_HEAD_INIT’
11 | PyVarObject_HEAD_INIT(NULL, 0)
it looks like I have to use a different mechanism to setup custom types in the
abi3 world.
I looked in Modules/xxlimited_35.c, but that seems much more complex and
provides for a type which supports GC.
Are there any ABI3 examples using the old style strategy?
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: is mypy failing here
On 24/11/2022 13:50, Kirill Ratkin via Python-list wrote: mypy --strict gives you detail info. Thanks Kirill, it seems --strict does find the errors. One of those is that on line 9 I have to add a return type ie def main() -> None: . if that is added then mypy without --strict also finds the real typing error. So it seems the tool fails in the simplest cases if you forget some typing. Interesting that it works in windows without --strict though. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: is mypy failing here
On 24/11/2022 14:10, Thomas Passin wrote: . C:\temp\python>py -V Python 3.10.4 C:\temp\python>py tdc.py DC(a=, b='B') C:\temp\python>mypy tdc.py tdc.py:10: error: Argument 1 to "DC" has incompatible type "Type[DC]"; expected "str" [arg-type] Found 1 error in 1 file (checked 1 source file) Forgot the mypy version: C:\Users\tom>mypy --version mypy 0.910 interesting; I'm on archlinux and neither the system python 3.10.8 / mypy 0.982 gives an error. I did try running in my self build 3.10.8 with latest mypy 0.991 and mypy 0.910 and I still don't get an error. I'll break out the windows 10 laptop and see what happens there. You ran with the py runner. I wonder if that does something special. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
is mypy failing here
I haven't used dataclasses or typing very much, but while playing about I found this didn't give me an expected error (.py312) robin@minikat:~/devel/reportlab $ cat tmp/examples/tdc.py && python tmp/examples/tdc.py && mypy tmp/examples/tdc.py ## from dataclasses import dataclass @dataclass class DC: a: str b: str def main(): dc = DC(DC, "B") print(dc) if __name__ == "__main__": main() ## DC(a=, b='B') Success: no issues found in 1 source file (.py312) robin@minikat:~/devel/reportlab DC.a is supposed to be a str and I expected mypy to indicate a type error should typing work for this case? -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Reportlab / platypus bug?
On 15/03/2022 13:20, Les wrote: Robin Becker ezt írta (időpont: 2022. márc. 15., K, 14:06): Hi Les, so far as I know the reportlab-users list is still running it is hosted (nad has been for many years) at https://pairlist2.pair.net/mailman/listinfo/reportlab-users is that the address you used? I see messages in the archives so some people can use it.-- Yes, it is. I tried to subscribe (twice), but never got the confirmation email. Checked in the spam folder too, but it is nowhere to be found. Hi again, Les. I just looked at the list using the admin login and find you are already a member nagy...@gmail.com Laszlo Nagy in fact you are number 1 in the N's. Perhaps you could just try sending a test email. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Reportlab / platypus bug?
.. Hi Les, so far as I know the reportlab-users list is still running it is hosted (nad has been for many years) at https://pairlist2.pair.net/mailman/listinfo/reportlab-users is that the address you used? I see messages in the archives so some people can use it.-- Robin Becker as a test I subscribed under my private email address and the list responded pretty quickly; the request confirmation email ended up in spam though. I believe the list is a fairly old version of mailman, but I don't have any access to the server. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Reportlab / platypus bug?
On 14/03/2022 18:17, Les wrote: Unfortunately, the reportlab-users mailing list is unavailable (I cannot subscribe). There is paid support but since I already have a workaround, I won't pay for this. I think this is a documentation error of the reportlab package. (They do not mention that stories cannot be reused.) I think we can say that my original problem is solved, because I have a workaround that always works. Schachner, Joseph ezt írta (időpont: 2022. márc. 14., H 19:09): Hi Les, so far as I know the reportlab-users list is still running it is hosted (nad has been for many years) at https://pairlist2.pair.net/mailman/listinfo/reportlab-users is that the address you used? I see messages in the archives so some people can use it.-- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: strange problem building non-pure wheel for apple M1 arm64
On 08/03/2022 16:08, Christian Gollwitzer wrote: Am 07.03.22 um 17:22 schrieb Robin Becker: I use brew to install freetype version 2.11.1. I find via google that homebrew/apple have split the installation of intel and arm64 into /usr/local and /opt/homebrew so I must modify the include_dirs & library_dirs in setup.py -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
strange problem building non-pure wheel for apple M1 arm64
I use cibuildwheel to build extensions with a github action. For the macos 11.0 arm64 build I get a strange message from the load command. So I am looking for assistance to try and figure out what is going wrong. The cibuild action uses the latest pip 21.2.4 and latest setuptools etc. I use brew to install freetype version 2.11.1. The compilations look like this gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -g -arch arm64 -DRENDERPM_FT -DLIBART_COMPILATION -DLIBART_VERSION=2.3.21 -Isrc/rl_addons/renderPM -Isrc/rl_addons/renderPM/libart_lgpl -Isrc/rl_addons/renderPM/gt1 -I/usr/local/include/freetype2 -I/Library/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c src/rl_addons/renderPM/_renderPM.c -o build/temp.macosx-11.0-arm64-3.9/src/rl_addons/renderPM/_renderPM.o this is the load command on multiple lines for readability the strange error is gcc -bundle -undefined dynamic_lookup -g -arch arm64 build/temp.macosx-11.0-arm64-3.9/src/rl_addons/renderPM/_renderPM.o build/temp.macosx-11.0-arm64-3.9/src/rl_addons/renderPM/gt1/gt1-dict.o build/temp.macosx-11.0-arm64-3.9/src/rl_addons/renderPM/gt1/gt1-namecontext.o '''other compiled code build/temp.macosx-11.0-arm64-3.9/src/rl_addons/renderPM/libart_lgpl/art_vpath_dash.o -L/usr/local/lib -L/usr/lib -L/Library/Frameworks/Python.framework/Versions/3.9/lib -lfreetype -o build/lib.macosx-11.0-arm64-3.9/reportlab/graphics/_renderPM.cpython-39-darwin.so ld: warning: ignoring file /usr/local/lib/libfreetype.dylib, building for macOS-arm64 but attempting to link with file built for macOS-x86_64 The above message seems bizarre; everything is compiled for arm64, but gcc doesn't want to use an arm64 dylib. Can macos experts assist? -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: lxml empty versus self closed tag
On 02/03/2022 18:39, Dieter Maurer wrote: Robin Becker wrote at 2022-3-2 15:32 +: I'm using lxml.etree.XMLParser and would like to distinguish from I seem to have e.getchildren()==[] and e.text==None for both cases. Is there a way to get the first to have e.text=='' I do not think so (at least not without a DTD): I have a DTD which has so I guess the empty case is allowed as well as the self closed. I am converting from an older parser which has text=='' for and text==None for the self closed version. I don't think I really need to make the distinction. However, I wonder how lxml can present an empty string content deliberately or if that always has to be a semantic decision. ` ag/>' is just a shorthand notation for '' and the difference has no influence on the DOM. Note that `lxml` is just a Python binding for `libxml2`. All the parsing is done by this library. yes I think I knew that -- https://mail.python.org/mailman/listinfo/python-list
lxml empty versus self closed tag
I'm using lxml.etree.XMLParser and would like to distinguish from I seem to have e.getchildren()==[] and e.text==None for both cases. Is there a way to get the first to have e.text=='' -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: preserving entities with lxml
On 13/01/2022 09:29, Dieter Maurer wrote: Robin Becker wrote at 2022-1-13 09:13 +: On 12/01/2022 20:49, Dieter Maurer wrote: ... Apparently, the `resolve_entities=False` was not effective: otherwise, your tree content should have more structure (especially some entity reference children). except that the tree knows not to expand the entities using ET.tostring so in some circumstances resolve_entities=False does work. I think this is a misunderstanding: `tostring` will represent the text character `&` as ``. aaa, thanks I see now. So tostring is actually restoring some of the entities which on input are normally expanded. If that means resolve_entities=False does not work at all then I guess there's no need to use it at all. The initial transform & --> does what I need as it is reversed on output of the tree fragments. Wonder what resolve_entities is actually used for then? All the docs seem to say resolve_entities - replace entities by their text value (default: True) I assumed False would mean that they would pass through the parse -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: preserving entities with lxml
On 12/01/2022 20:49, Dieter Maurer wrote: ... when run I see this $ python tmp/tlp.py using tostring xxml=b'a mysym; lt; amp; gt; #33; A' ET.tostring(tree)=b'a mysym; lt; amp; gt; #33; A' using attributes tree.text='a A' tree.getchildren()=[] tree.tail=None Apparently, the `resolve_entities=False` was not effective: otherwise, your tree content should have more structure (especially some entity reference children). except that the tree knows not to expand the entities using ET.tostring so in some circumstances resolve_entities=False does work. I expected that the tree would contain the parsed (unexpanded) values, but referencing the actual tree.text/tail/attrib doesn't give the expected results. There's no criticism here, it makes my life a bit easier. If I had wanted the unexpanded values in the attrib/text/tail it would be more of a problem.
preserving entities with lxml
I have a puzzle over how lxml & entities should be 'preserved' code below illustrates. To preserve I change & -->
in the source and add resolve_entities=False to the parser definition. The escaping means we only have one kind of
entity which means lxml will preserve it. For whatever reason lxml won't preserve character entities eg .
The simple parse from string and conversion tostring shows that the parsing at
least took notice of it.
However, I want to create a tuple tree so have to use tree.text,
tree.getchildren() and tree.tail for access.
When I use those I expected to have to undo the escaping to get back the original entities, but it seems they are
already done.
Good for me, but if the tree knows how it was created (tostring shows that) why
is it ignored with attribute access?
if __name__=='__main__':
from lxml import etree as ET
#initial xml
xml = b'a
A'
#escaped xml
xxml = xml.replace(b'&',b'')
myparser = ET.XMLParser(resolve_entities=False)
tree = ET.fromstring(xxml,parser=myparser)
#use tostring
print(f'using tostring\n{xxml=!r}\n{ET.tostring(tree)=!r}\n')
#now access the items using text & children & text
print(f'using
attributes\n{tree.text=!r}\n{tree.getchildren()=!r}\n{tree.tail=!r}')
when run I see this
$ python tmp/tlp.py
using tostring
xxml=b'a mysym; lt; amp; gt;
#33; A'
ET.tostring(tree)=b'a mysym; lt; amp;
gt; #33; A'
using attributes
tree.text='a A'
tree.getchildren()=[]
tree.tail=None
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: lxml parsing with validation and target?
On 02/11/2021 12:55, Robin Becker wrote: I'm having a problem using lxml.etree to make a treebuilding parser that validates; I have test code where invalid xml is detected and an error raised when the line below target=ET.TreeBuilder(), is commented out. . I managed to overcome this problem by utilizing the non-targeted parser with returns an _ElementTree object. I can then convert to tuple tree using code like this class TT: def __init__(self): pass def __call__(self,tree): if not tree: return return self.maketuple(next(tree.iter())) def maketuple(self,e): return (e.tag,e.attrib or None,self.content(e),e.sourceline) def content(self,e): t = e.text kids = e.getchildren() if len(kids)==0 and t is None: return t else: r = [].append if t is not None: r(t) for c in kids: r(self.maketuple(c)) t = c.tail if t is not None: r(t) return r.__self__ -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
lxml parsing with validation and target?
I'm having a problem using lxml.etree to make a treebuilding parser that validates; I have test code where invalid xml
is detected and an error raised when the line below target=ET.TreeBuilder(), is commented out.
The validation error looks as expected > python tlxml.py invalid.rml
re.compile('^.*(?:\\W|\\b)(?Pdynamic_rml\\.dtd|rml\\.dtd|rml_0_2\\.dtd|rml_0_3\\.dtd|rml_1_0\\.dtd)$',
re.MULTILINE)
Resolving url='../rml.dtd' context= dtdPath='rml.dtd'
Traceback (most recent call last):
File "/home/robin/devel/reportlab/REPOS/rlextra/tmp/tlxml.py", line 78, in
tree = ET.parse(sys.argv[1],parser)
File "src/lxml/etree.pyx", line 3521, in lxml.etree.parse
File "src/lxml/parser.pxi", line 1859, in lxml.etree._parseDocument
File "src/lxml/parser.pxi", line 1885, in lxml.etree._parseDocumentFromURL
File "src/lxml/parser.pxi", line 1789, in lxml.etree._parseDocFromFile
File "src/lxml/parser.pxi", line 1177, in
lxml.etree._BaseParser._parseDocFromFile
File "src/lxml/parser.pxi", line 615, in
lxml.etree._ParserContext._handleParseResultDoc
File "src/lxml/parser.pxi", line 725, in lxml.etree._handleParseResult
File "src/lxml/parser.pxi", line 654, in lxml.etree._raiseParseError
File "invalid.rml", line 23
lxml.etree.XMLSyntaxError: No declaration for attribute x of element place1,
line 23, column 55
when I have the target=etree.TreeBuilder() active the validation does not work and the tree is formed and passed to the
primitive tuple tree builder so the output looks like
$ python tlxml.py invalid.rml
Resolving url='../rml.dtd' context= dtdPath='rml.dtd'
('document',
{'filename': 'test_000_simple.pdf', 'invariant': '1'},
['\n\n',
('stylesheet',
>
None,
44),
'\n\t\t\n\t\t'],
40),
'\n'],
35),
'\n\n'],
2)
If I use the standard example EchoTarget the validation also fails. So I assume that the target argument makes the
validation fail. Is there a way to get validation to work with a target?
The code is
##
from pprint import pprint
from lxml import etree as ET
import sys, os, re
from rlextra.rml2pdf.rml2pdf import CompatibleDTDNames as rmlDTDPat
rmlDTDPat = re.compile('^.*(?:\\W|\\b)(?P%s)$' % '|'.join((re.escape(_) for
_ in rmlDTDPat)),re.M)
class TT:
def __init__(self):
pass
def __call__(self,e):
return (e.tag,e.attrib or None,self.content(e),e.sourceline)
def content(self,e):
t = e.text
if len(e)==0 and t is None:
return t
else:
r = [].append
if t is not None: r(t)
for c in e:
r(self(c))
t = c.tail
if t is not None:
r(t)
return r.__self__
class RMLDTDResolver(ET.Resolver):
__dtds = None
def resolve(self, url, id, context):
m = rmlDTDPat.match(url)
if m:
if self.__dtds is None:
from rlextra import rml2pdf
self.__dtds = {}
for fn in ('rml.dtd','dynamic_rml.dtd'):
with
open(os.path.join(os.path.dirname(rml2pdf.__file__),fn),'r') as _:
self.__dtds[fn] = _.read()
fn = m.group('fn')
dtdPath = 'rml.dtd' if fn.startswith('rml') else 'dynamic.dtd'
print(f"Resolving url={url!r} context={context!r} {dtdPath=}")
return self.resolve_string(
self.__dtds[dtdPath],
context,
)
else:
return None
parser = ET.XMLParser(
load_dtd=True,
dtd_validation=True,
attribute_defaults=True,
no_network=True,
remove_comments=True,
remove_pis=True,
strip_cdata=True,
resolve_entities=True,
target=ET.TreeBuilder(), #if commented the parser
validates
)
parser.resolvers.add(RMLDTDResolver())
tree = ET.parse(sys.argv[1],parser)
pprint(TT()(tree))
##
--
https://mail.python.org/mailman/listinfo/python-list
Re: c extension finding the module in object initialization
Hi Marc, Thanks for the suggestion, On 27/09/2021 09:38, Marc-Andre Lemburg wrote: Hi Robin, seeing that no one replied to your question, I'd suggest to ask this on the Python C-API ML: https://mail.python.org/mailman3/lists/capi-sig.python.org/ That's where the experts are, including the ones who implemented the mutli-phase logic. Cheers, I think I have this working using ob=PyImport_GetModuleDict() followed by PyDict_GetItemString(ob,"modulename"), but I will ask there to see if there's a more direct route. In Python >=3.7 there's PyImport_GetModule, but that seems more complex than is actually required for this simple case and has to wait until 3.6 dies for me :( -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
c extension finding the module in object initialization
I have a c extension which is intended to implement a module the looks structurally like this a = 1 b = 2 class T: def __init__(self): self.a = a self.b = b so when an object of type T is instantiated it can set up defaults based on the current module values of a and b. In the past using old style single phase module creation the init function for the type could look up the module by using the PyState_FindModule function. In the new world where we can implement c extensions which might work with multiple interpreters (multi-phase creation). The docs say PyState_FindModule won't work for those and indeed it returns NULL so is useless. Is there a way I can set the current module onto the type definition during the module's exec function? I thought it would be easy to add at the point in the exec where I'm doing this to the module, m, TType.tp_base = _Type; if(PyType_Ready()<0) goto fail; if(PyModule_AddObject(m,"T", (PyObject *))<0) goto fail; but I don't see the place in the type where I can add these sorts of dynamic attributes. The basic_size is for the created objects. The created type does have a __dict__ which is a mappingproxy. I wondered when that actually gets instantiated. I can see that the type has a __module__ attribute after the module is imported and I suppose if I can find the 'current' interpreter I might use the __module__ to locate the module object via PyImport_GetModuleDict. Any expertise or advice gratefully received. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
what to do with multiple BOMs
Channeling unicode text experts and xml people: I have xml entity with initial bytes ff fe ff fe which the file command says is UTF-16, little-endian text. I agree, but what should be done about the additional BOM. A test output made many years ago seems to keep the extra BOM. The xml context is xml file 014.xml ]> \xef\xbb\xbfdata' which implies seems as though the extra BOM in the entity has been kept and processed into a different BOM meaning utf8. I think the test file is wrong and that multiple BOM chars in the entiry should have been removed. Am I right? -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: basic auth request
On 17/08/2021 22:47, Jon Ribbens via Python-list wrote: ... That's only true if you're not using HTTPS - and you should *never* not be using HTTPS, and that goes double if forms are being filled in and double again if passwords are being supplied. I think I agree with most of the replies; I understood from reading the rfc that the charset is utf8 (presumably without ':') and that basic auth is considered insecure. It is being used over https so should avoid the simplest net scanning. I googled a bunch of ways to do this, but many come down to 1) using the requests package or 2) setting up an opener. Both of these seem to be much more complex than is required to add the header. I thought there might be a shortcut or more elegant way to replace the old code, but it seems not thanks -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
basic auth request
While porting an ap from python2.7 to python3 I see this
base64string = base64.b64encode('%s:%s' % (wsemail, wspassword))
request.add_header("Authorization", "Basic %s" % base64string)
in python3.x I find this works
base64string = base64.b64encode(('%s:%s' % (wsemail,
wspassword)).encode('ascii')).decode('ascii')
request.add_header("Authorization", "Basic %s" % base64string)
but I find the conversion to and from ascii irksome. Is there a more direct way
to create the basic auth value?
As an additional issue I find I have no clear idea what encoding is allowed for
the components of a basic auth input.
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
error from pypa build
I'm building a pure python wheel in a python3.9 virtualenv. If I use python setup.py bdist_wheel I do get a wheel named rlextra-3.6.0-py3-none-any.whl I'm trying out building a modern python 3 only wheel so I followed instructions here https://packaging.python.org/tutorials/packaging-projects/#creating-the-package-files and see the error below which is not very informative. Any ideas of what's wrong? I still get the error even after upgrading pip so I don't think that is the problem. > (.py39) user@pc:~/devel/reportlab/REPOS/rlextra > $ pip install build > Collecting build > Downloading build-0.5.1-py2.py3-none-any.whl (15 kB) > Collecting pep517>=0.9.1 > .. > Installing collected packages: tomli, pyparsing, toml, pep517, packaging, build > Successfully installed build-0.5.1 packaging-21.0 pep517-0.11.0 pyparsing-2.4.7 toml-0.10.2 tomli-1.2.0 > WARNING: You are using pip version 21.1.3; however, version 21.2.2 is available. > You should consider upgrading via the '/home/user/devel/reportlab/.py39/bin/python39 -m pip install --upgrade pip' command. > (.py39) user@pc:~/devel/reportlab/REPOS/rlextra > $ ls > build dist docs examples LICENSE.txt README.txt setup.cfg setup.py src tests tmp > (.py39) user@pc:~/devel/reportlab/REPOS/rlextra > $ python -m build --wheel > Traceback (most recent call last): > File "/home/user/devel/reportlab/.py39/lib/python3.9/site-packages/build/__main__.py", line 302, in main > build_call(args.srcdir, outdir, distributions, config_settings, not args.no_isolation, args.skip_dependency_check) > File "/home/user/devel/reportlab/.py39/lib/python3.9/site-packages/build/__main__.py", line 145, in build_package > _build(isolation, builder, outdir, distribution, config_settings, skip_dependency_check) > File "/home/user/devel/reportlab/.py39/lib/python3.9/site-packages/build/__main__.py", line 101, in _build > return _build_in_isolated_env(builder, outdir, distribution, config_settings) > File "/home/user/devel/reportlab/.py39/lib/python3.9/site-packages/build/__main__.py", line 77, in _build_in_isolated_env > with IsolatedEnvBuilder() as env: > File "/home/user/devel/reportlab/.py39/lib/python3.9/site-packages/build/env.py", line 92, in __enter__ > executable, scripts_dir = _create_isolated_env_venv(self._path) > File "/home/user/devel/reportlab/.py39/lib/python3.9/site-packages/build/env.py", line 221, in _create_isolated_env_venv > pip_distribution = next(iter(metadata.distributions(name='pip', path=[purelib]))) > StopIteration > > ERROR > (.py39) user@pc:~/devel/reportlab/REPOS/rlextra thanks for any assistance -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Python script accessing own source code
On 12/05/2021 20:17, Mirko via Python-list wrote: Am 12.05.2021 um 20:41 schrieb Robin Becker: ... ... since GvR has been shown to have time traveling abilities such a script could paradoxically appear acausally. -- yrs-not-too-seriously Robin Becker Not sure, if that's what you mean, but writing a self-replicating script is easy too: actually I was really joking about self creating scripts that do something useful like finding future lotto numbers. the artificial programmer servant is some way off import os import sys with open(os.path.abspath(__file__)) as myself: with open(sys.argv[1], "w") as yourself: yourself.write(myself.read()) Give it a filename as a command-line argument and it will write itself to that file. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Python script accessing own source code
... with open(__file__) as myself: print(myself.read(), end='') very nice, but accessing code that's already seems quite easy. I think the real problem is to get a python script name that creates and writes itself. So I would ask if any one has the solution to the self writing script python find-tomorrows-lotto-numbers.py since GvR has been shown to have time traveling abilities such a script could paradoxically appear acausally. -- yrs-not-too-seriously Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
new match statement and types
I'm trying out new features in 3.10.0a6 looking at the new match statement I tried it with various cases. Knowing that
the class of a class is 'type' I tried using match to distinguish between classes and instances so I tried various
versions of
###
class A:
pass
class B:
pass
def tmc(C):
match C:
case type(__name__='A'):
print(f'{C} is an A' )
case type():
print(f'{C} is a type' )
case A():
print(f'{C} is an A instance')
case _:
print(f'{C} is an instance')
if __name__=='__main__':
tmc(A)
tmc(B)
tmc(A())
tmc(B())
###
the above seems to work and produces
is an A
is a type
<__main__.A object at 0x7fe5a2248fd0> is an A instance
<__main__.B object at 0x7fe5a2248fd0> is an instance
is this the right approach to this problem of distinguishing instances ?
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: idlelib re-use
Thanks, On 28/01/2021 19:57, Terry Reedy wrote: On 1/28/2021 5:53 AM, Robin Becker wrote: I googled in vain for instances where parts of idlelib are re-used in a simplistic way. I would like to use the editor functionality in a tkinter window and also probably run code in a subprocess. Are there any examples around that do these sorts of things? turtledemo reuses IDLE's colorizer and read-only textviews. I have seen occasional hints on stackoverflow of other such uses. One barrier to reuse is that the parts are highly interconnected, with numerous import loops. (Changing the form of some imports from 'import x' to 'from x import y' can make IDLE startup fail.) Some objects, like EditorWindow, are too monolithic. You cannot put a toplevel inside another toplevel. yes I found this out trying to reuse the editor window and shell. Another, for those thinking long term, is that implementation modules in idlelib are defined as private since 3.6 (PEP-434, idlelib.__init__). I pushed for this in order to be able to refactor to reduce interconnections, and break some things apart, and also to switch to ttk widgets. For instance, breaking EditorFrame apart from EditorWindow would allow me to put multiple editors on multiple tabs of a notebook in an application window. It would also allow others to put an editor window in their tkinter window. +1 -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
idlelib re-use
I googled in vain for instances where parts of idlelib are re-used in a simplistic way. I would like to use the editor functionality in a tkinter window and also probably run code in a subprocess. Are there any examples around that do these sorts of things? -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: advice on debugging a segfault
On 17/01/2021 21:35, Stestagg wrote: I would normally agree, except... This is a refcount issue (I was able to reproduce the problem, gbd shows a free error ) And I wouldn't recommend DGBing a refcount issue as a beginner to debugging. The other mailing list identified a PIL bug that messes up the refcount for True, but this refcount issue is for some tuple object, so may possibly be a different problem. Steve .. thanks for all the advice and apologies for the noise. It turns out that Victor Stinner's recommendation in the dev list was the solution as I don't see the segfault with pillow built from latest git Pillow @ git+git://github.com/python-pillow/pillow.git@8dc5f692dbd9a418d8c441f0e2aa09a3f5c03508 I did realize that this must be a late issue in the process as the pdf was being produced and written as expected and that is normally the end of things -- not segfaulting-ly yrs Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
advice on debugging a segfault
I have a segfault in the 3.10 alpha 4 when running the reportlab document generation; all the other tests seem to have worked. I would like to ask experts here how to approach getting the location of the problem. I run recent archlinux. Below is the output of a test run with -Xdev -Xtracemalloc; the main process is almost finished so the error appears to come from trying to free resources $ python -Xdev -Xtracemalloc genuserguide.py /home/robin/devel/reportlab/.py310/lib/python3.10/distutils/__init__.py:1: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses import imp Built story contains 1843 flowables... Saved "/home/robin/devel/reportlab/REPOS/reportlab/docs/reportlab-userguide.pdf" [ Top 10 ] :630: size=10.5 MiB, count=105832, average=104 B /home/robin/devel/reportlab/reportlab/lib/abag.py:19: size=7161 KiB, count=27126, average=270 B /home/robin/devel/reportlab/reportlab/platypus/paraparser.py:3141: size=3364 KiB, count=5980, average=576 B /home/robin/devel/reportlab/.py310/lib/python3.10/site-packages/pyphen/__init__.py:160: size=2905 KiB, count=41797, average=71 B /home/robin/devel/reportlab/reportlab/platypus/paragraph.py:776: size=1386 KiB, count=32208, average=44 B /home/robin/devel/reportlab/reportlab/platypus/paragraph.py:92: size=1384 KiB, count=26248, average=54 B /home/robin/devel/reportlab/.py310/lib/python3.10/copy.py:280: size=1232 KiB, count=8827, average=143 B /home/robin/devel/reportlab/reportlab/platypus/frames.py:155: size=1101 KiB, count=1173, average=962 B :241: size=915 KiB, count=8146, average=115 B /home/robin/devel/reportlab/reportlab/platypus/paragraph.py:773: size=881 KiB, count=16104, average=56 B sys:1: ResourceWarning: unclosed file <_io.BufferedReader name='../images/replogo.gif'> sys:1: ResourceWarning: unclosed file <_io.BufferedReader name='../images/testimg.gif'> Debug memory block at address p=0x5617c33effe0: API '' 0 bytes originally requested The 7 pad bytes at p-7 are not all FORBIDDENBYTE (0xfd): at p-7: 0x00 *** OUCH at p-6: 0x00 *** OUCH at p-5: 0x00 *** OUCH at p-4: 0x00 *** OUCH at p-3: 0x00 *** OUCH at p-2: 0x00 *** OUCH at p-1: 0x00 *** OUCH Because memory is corrupted at the start, the count of bytes requested may be bogus, and checking the trailing pad bytes may segfault. The 8 pad bytes at tail=0x5617c33effe0 are not all FORBIDDENBYTE (0xfd): at tail+0: 0x00 *** OUCH at tail+1: 0x00 *** OUCH at tail+2: 0x00 *** OUCH at tail+3: 0x00 *** OUCH at tail+4: 0x00 *** OUCH at tail+5: 0x00 *** OUCH at tail+6: 0x00 *** OUCH at tail+7: 0x00 *** OUCH Enable tracemalloc to get the memory block allocation traceback Fatal Python error: _PyMem_DebugRawFree: bad ID: Allocated using API '', verified using API 'o' Python runtime state: finalizing (tstate=0x5617c53c8b30) Current thread 0x7fd5742b3740 (most recent call first): Aborted (core dumped) for comparison here is the 3.9.1 output> $ python -Xdev -Xtracemalloc genuserguide.py /home/robin/devel/reportlab/.py39/lib/python3.9/distutils/__init__.py:1: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses import imp /home/robin/devel/reportlab/.py39/lib/python3.9/site-packages/fontTools/misc/py23.py:11: DeprecationWarning: The py23 module has been deprecated and will be removed in the next release. Please update your code. warnings.warn( Built story contains 1843 flowables... Saved "/home/robin/devel/reportlab/REPOS/reportlab/docs/reportlab-userguide.pdf" [ Top 10 ] :587: size=12.4 MiB, count=128010, average=102 B /home/robin/devel/reportlab/reportlab/lib/abag.py:19: size=7132 KiB, count=26951, average=271 B /home/robin/devel/reportlab/reportlab/platypus/paraparser.py:3141: size=3364 KiB, count=5980, average=576 B /home/robin/devel/reportlab/.py39/lib/python3.9/site-packages/pyphen/__init__.py:160: size=2905 KiB, count=41797, average=71 B /home/robin/devel/reportlab/reportlab/platypus/paragraph.py:776: size=1386 KiB, count=32208, average=44 B /home/robin/devel/reportlab/reportlab/platypus/paragraph.py:92: size=1384 KiB, count=26248, average=54 B /home/robin/devel/reportlab/.py39/lib/python3.9/copy.py:279: size=1230 KiB, count=8827, average=143 B /home/robin/devel/reportlab/reportlab/platypus/frames.py:155: size=1039 KiB, count=957, average=1112 B :228: size=959 KiB, count=8503, average=116 B /home/robin/devel/reportlab/reportlab/platypus/paragraph.py:773: size=881 KiB, count=16104, average=56 B sys:1: ResourceWarning: unclosed file <_io.BufferedReader name='../images/replogo.gif'> sys:1: ResourceWarning: unclosed file <_io.BufferedReader name='../images/testimg.gif'> -- https://mail.python.org/mailman/listinfo/python-list
Re: Finding it very difficult to move pyexiv2 code from Python 2 to Python 3
. so obviously I need to install some version of boost libs or Boost.Python etc etc. Gave up :( -luddite-ly yrs- Robin Becker The aur repository, no ? https://aur.archlinux.org/packages/python-exiv2/ Vincent that would work (if I had thought hard about it), but not for a pip install which is where the fun started :), I was only trying to see what was going wrong. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Finding it very difficult to move pyexiv2 code from Python 2 to Python 3
On 18/08/2020 20:05, Vincent Vande Vyvre wrote: . Hi, Two solutions: 1. Install exiv2-dev and py3exiv2 with pip $ sudo apt-get install libexiv2-dev $ sudo pip3 install py3exiv2 2. Install my ppa $ sudo add-apt-repository ppa:vincent-vandevyvre/vvv $ sudo apt-get update $ sudo apt-get install python3-exiv2 Don't change your old code for pyexiv2, the names of the modules are unchanged, your old code should work as it. Off course old strings are now unicode. Vincent (AKA VinsS) I haven't tried #2 (I use Arch), but I did try 1) in a python 3.8 virtual env. I ended up getting into a problem with boost src/exiv2wrapper.hpp:34:10: fatal error: boost/python.hpp: No such file or directory 34 | #include "boost/python.hpp" | ^~ compilation terminated. error: command 'gcc' failed with exit status 1 so obviously I need to install some version of boost libs or Boost.Python etc etc. Gave up :( -luddite-ly yrs- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Final statement from Steering Council on politically-charged commit messages
On 18/08/2020 19:45, Tim Daneliuk wrote: On 8/18/20 12:28 PM, justin walters wrote: I apologize for being ageist earlier as well. That was out of line. I am likely older than you and there is no reason to apologise. Only the profoundly undeveloped psyche takes every opportunity to find offense when none is intended. It is the sign of a puerile mind, irrespective of actual chronological age. Feel free to be as "ageist" as you wish ... only make it funny or biting ... +10 I too am of a previous generation that recited "sticks and stones .". I find the hyper-sensitivity of the woken rather dispiriting. -aged-ly yrs- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Final statement from Steering Council on politically-charged commit messages
On 18/08/2020 04:53, thronobu...@gmail.com wrote: "Truth is a social construct." much as I deplore the politicization of computers, logic, maths and other areas of human interest by particular interest groups, according to some physicists, reality might be an observer based construct https://www.sciencemag.org/news/2020/08/quantum-paradox-points-shaky-foundations-reality ReportLab has quite a lot of colour based words; so far I've only had a few related emails :) which mostly seem to end up in spam -hoping to escape victimhood-ly yrs- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Are instances of user-defined classes mutable?
On 06/08/2020 05:17, ZHAOWANCHENG wrote:
the doc of dictionary said "if a tuple contains any mutable object either directly
or indirectly, it cannot be used as a key."
i think a instance of user-defined class is mutable, but i found it can be
placed into a tuple that as a key of a dict:
>>> class mycls(object):
... a = 1
...
>>> me = mycls()
>>> me.a = 2 # mutable?
>>> {(1, me): 'mycls'}
{(1, <__main__.mycls object at 0x022824DAD668>): 'mycls'}
>>>
So are instances of user-defined classes mutable or immutable?
user class instances are clearly mutable, and in my python 3.8 you can do
horrid things like this
class H:
... a = 1
... def __hash__(self):
... return hash(self.a)
...
h = H()
hash(h)
1
h.a =2
hash(h)
2
t=(1,h)
d={t:23}
d
{(1, <__main__.H object at 0x7f5bf72021f0>): 23}
hash(h)
2
hash(list(d.keys())[0])
-3550055125485641917
h.a=33
hash(list(d.keys())[0])
-3656087029879219665
so the dict itself doesn't enforce immutability of its keys
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: Formal Question to Steering Council (re recent PEP8 changes)
Since explicit is better than implicit :-), I would like to formally ask the Steering Council to answer the following questions. 1. Does the Steering Council think political statements have any place in the Python repositories? 2. If so, for the avoidance of doubt does the Steering Council support the statements in commit 0c6427d? (https://github.com/python/peps/commit/0c6427dcec1e98ca0bd46a876a7219ee4a9347f4) 3. If not, what do they intend to do about the above commit? If the answer to question 1 is a qualified yes or no, both follow-up questions apply. If the answer to question 1 is a prevarication, non-answer or silence, people will still draw their own conclusions. I mention this merely to reinforce the idea that these things are still answers as well as hostages to fortune. the above gets +10 from me according to the telegraph John Cleese has accused the BBC of “social engineering” after its head of comedy said Monty Python’s white Oxbridge males were out of step with modern television. so is there a pep for alternate language names ;) -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: pyinstaller
On 11/06/2020 16:39, Grant Edwards wrote: the hands of the developer. I suppose the OP could quit and stand on the street corner with a cardboard sign: I would love to do that :) -- https://mail.python.org/mailman/listinfo/python-list
Re: pyinstaller
.. Submit the .exe to Microsoft so they can update the virus definitions to remove your false positive. https://www.microsoft.com/en-us/wdsi/filesubmission <https://www.microsoft.com/en-us/wdsi/filesubmission> I did this for one of my open source projects and Microsoft fixed the false problem in a few days for me. I think my boss actually tried this, but I will check. Thanks Barry .... -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: pyinstaller
On 11/06/2020 09:24, Chris Angelico wrote: But you DO get to choose what you support. If they want to use Windows 98 on an 80386, that's their (stupid) choice, and they can't guarantee that your program will work on it. sadly this isn't really true since this is only part of the process and we are trying to capture data (sports results) which they create on their hardware for other reasons; some machines are 20 years old and some this year. Some of the capture devices are using serial ports :( which luckily I don't get near :) Regarding the users as stupid because they won't follow us on the upgrade tread wheel seems a bit arrogant. ChrisA -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: pyinstaller
On 10/06/2020 20:41, Chris Angelico wrote: . Python can be installed from the app store, or from a python.org downloader. If that's too much hassle for them, then they're going to need help *whatever* you do. works for windows 10, but probably not on older machines. The current exe works on very old versions of windows as well. Some people are reluctant to change old win 95/xp machines just to run a single app. And a lot of us are reluctant to try to support XP. It's an operating system that was released the same year as Python 2.2 - do you try to support that? If not, why support an ancient OS? unfortunately we don't get to choose the users' hardware or what OS they use. I had supposed there might be a simple mechanism to get these applications validated in some way, but it seems not. MS seems uninterested. And I can't blame them. How is MS going to know that you haven't tampered with the Python binary before you packaged it up? How can end users be expected to trust it? no disagreement here ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: pyinstaller
On 10/06/2020 15:02, Chris Angelico wrote: The intended users are unlikely to understand how to adjust the scanner to whitelist the application. Tell them to install Python from an official source, and then distribute your application as a single .py (or .pyw) file. Problem solved. ChrisA Thanks for the obvious suggestion, but if they have difficulty whitelisting the app then installing python is also probably beyond their abilities :( and I think it ought to be easier for the pyinstaller created executables to 'just work'. The current exe works on very old versions of windows as well. Some people are reluctant to change old win 95/xp machines just to run a single app. I had supposed there might be a simple mechanism to get these applications validated in some way, but it seems not. MS seems uninterested. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: pyinstaller
On 10/06/2020 16:11, Souvik Dutta wrote: I found this... https://stackoverflow.com/questions/43777106/program-made-with-pyinstaller-now-seen-as-a-trojan-horse-by-avg Might be usefull, might be useless. Souvik flutter dev On Wed, Jun 10, 2020, 7:18 PM Robin Becker wrote: . thanks for the above. I might try to do a downgrade as that is suggested as one of the ways forward. I don't want to start compiling my own versions of the boostrap exe bits. Clearly most virus scanners only look at the obvious behaviour ie we execute a lot of python code which is probably impenetrable to the scanner and hence suspicious. I submitted my exe to the web based scanner in that answer and it came out clean, but windows 10 barks :( -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: pyinstaller
On 10/06/2020 15:18, Souvik Dutta wrote: You might also try py2exe, in that way the user doesn't need to install python in her/his computer. Souvik flutter dev On Wed, Jun 10, 2020, 7:18 PM Robin Becker wrote: I'm sure this has come up before, but a tiny pyinstaller created exe is being seen as malware by windows 10. I think we used py2exe previously and I guess that will have the same problems as pyinstaller. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
pyinstaller
I'm sure this has come up before, but a tiny pyinstaller created exe is being seen as malware by windows 10. Is there any way to create simple single file applications which don't get this treatment? The intended users are unlikely to understand how to adjust the scanner to whitelist the application. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: [RELEASE] Python 3.9.0b1 is now available for testing
I used https://github.com/python/pyperformance pyperformance to compare Arch linux latest Python 3.8.3 (default, May 17 2020, 18:15:42) [GCC 10.1.0] on linux Type "help", "copyright", "credits" or "license" for more information. against a vanilla build (configure make makeinstall) of python 3.9b1 Python 3.9.0b1 (default, May 19 2020, 21:09:14) [GCC 10.1.0] on linux Type "help", "copyright", "credits" or "license" for more information. I find all the bench marks seem to be slower in python 3.9b1. 38.json === Performance version: 1.0.1 Report on Linux-5.6.14-arch1-1-x86_64-with-glibc2.2.5 Number of logical CPUs: 4 Start date: 2020-05-31 04:00:24.503704 End date: 2020-05-31 04:22:44.961331 39.json === Performance version: 1.0.1 Report on Linux-5.6.14-arch1-1-x86_64-with-glibc2.31 Number of logical CPUs: 4 Start date: 2020-05-31 04:23:21.247268 End date: 2020-05-31 04:49:09.891889 ### 2to3 ### Mean +- std dev: 437 ms +- 5 ms -> 548 ms +- 7 ms: 1.25x slower Significant (t=-96.22) ### chameleon ### Mean +- std dev: 12.5 ms +- 0.1 ms -> 16.2 ms +- 0.2 ms: 1.30x slower Significant (t=-111.53) > ... Is this because I haven't built in the same way as Arch or are there real slowdowns in this beta? Or even dumber have I got the results the wrong way round? -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: [RELEASE] Python 3.9.0b1 is now available for testing
On 19/05/2020 23:41, Robin Becker wrote:
..
robin@minikat:~/devel/reportlab
$ $HOME/LOCAL/3.9b1/bin/python3.9
Python 3.9.0b1 (default, May 19 2020, 12:50:30) [GCC 10.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
import distutils
/home/robin/LOCAL/3.9b1/lib/python3.9/distutils/__init__.py:15: UserWarning: The virtualenv distutils package at %s appears to be in the same location as the system distutils?
warnings.warn("The virtualenv distutils package at %s appears to be in the same
location as the system distutils?")
distutils.__path__
['/home/robin/LOCAL/3.9b1/lib/python3.9/distutils']
distutils.__file__
'/home/robin/LOCAL/3.9b1/lib/python3.9/distutils/__init__.py'
it seems installing virtualenv 16.2.0 into my python3.9 and then using it to create a venv modifies the base
distutils/__init__.py so the problem seems to lie with virtualenv (or at least the older version).
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: [RELEASE] Python 3.9.0b1 is now available for testing
..
robin@minikat:~/devel/reportlab
$ $HOME/LOCAL/3.9b1/bin/python3.9
Python 3.9.0b1 (default, May 19 2020, 12:50:30) [GCC 10.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
import distutils
/home/robin/LOCAL/3.9b1/lib/python3.9/distutils/__init__.py:15: UserWarning:
The virtualenv distutils package at %s appears to be in the same location as
the system distutils?
warnings.warn("The virtualenv distutils package at %s appears to be in the same
location as the system distutils?")
distutils.__path__
['/home/robin/LOCAL/3.9b1/lib/python3.9/distutils']
distutils.__file__
'/home/robin/LOCAL/3.9b1/lib/python3.9/distutils/__init__.py'
is this a bug or have I built python 3.9.0b1 wrongly?
I think that is correct __file__ value. What are you expecting to see?
I wasn't expecting the warning at all, but in fact this appears to be an interaction caused by installing
virualenv==16.2.0 into my home built python3.9.0b1 and then using it to create a virtual environment. For some reason
that creates a problem in the base python which I haven't yet fully explained.
I can build python3.9.9b1 virtualenvs using archlinux python 3.8.2 with
/bin/python -mvirtualenv -p /home/robin/LOCAL/3.9b1/bin/python3.9 myenv
I prefer the older virtualenv because it makes python executables which I can link to rather than always using an
explicit path.
Sorry for the noise.
Barry
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
O
--
https://mail.python.org/mailman/listinfo/python-list
Re: [RELEASE] Python 3.9.0b1 is now available for testing
On 19/05/2020 10:49, Łukasz Langa wrote:
On behalf of the entire Python development community, and the currently serving
Python release team in particular, I’m pleased to announce the release of
Python 3.9.0b1. Get it here:
https://www.python.org/downloads/release/python-390b1/
<https://www.python.org/downloads/release/python-390b1/>
This is a beta preview of Python 3.9
Python 3.9 is still in development. This release, 3.9.0b1, is the first of four
planned beta release previews.
Beta release previews are intended to give the wider community the opportunity
to test new features and bug fixes and to prepare their projects to support the
new feature release.
Call to action
.
I built python3.9.0b1 from source using the standard configure make dance eg
./configure --prefix=$HOME/LOCAL/3.9b1
make && make install
that appeared to work fine and I seem to get a working python. However, I seem
to have an issue with the distutils package
robin@minikat:~/devel/reportlab
$ $HOME/LOCAL/3.9b1/bin/python3.9
Python 3.9.0b1 (default, May 19 2020, 12:50:30)
[GCC 10.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
import distutils
/home/robin/LOCAL/3.9b1/lib/python3.9/distutils/__init__.py:15: UserWarning:
The virtualenv distutils package at %s appears to be in the same location as
the system distutils?
warnings.warn("The virtualenv distutils package at %s appears to be in the same
location as the system distutils?")
distutils.__path__
['/home/robin/LOCAL/3.9b1/lib/python3.9/distutils']
distutils.__file__
'/home/robin/LOCAL/3.9b1/lib/python3.9/distutils/__init__.py'
is this a bug or have I built python 3.9.0b1 wrongly?
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: [RELEASE] Python 3.9.0a6 is now available for testing
On 28/04/2020 16:52, Łukasz Langa wrote: On behalf of the entire Python development community, and the currently serving Python release team in particular, I’m pleased to announce the release of Python 3.9.0a6. Get it here: thanks for the release; I tried to reply in the dev list, but failed miserably. Sorry for any noise. I see this simple difference which broke some ancient code which works in Python 3.8.2 $ python Python 3.8.2 (default, Apr 8 2020, 14:31:25) [GCC 9.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. norm=lambda m: m+(m and(m[-1]!='\n'and'\n'or'')or'\n') robin@minikat:~/devel/reportlab/REPOS/reportlab/tests $ python39 Python 3.9.0a6 (default, Apr 29 2020, 07:46:29) [GCC 9.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. norm=lambda m: m+(m and(m[-1]!='\n'and'\n'or'')or'\n') File "", line 1 norm=lambda m: m+(m and(m[-1]!='\n'and'\n'or'')or'\n') ^ SyntaxError: invalid string prefix robin@minikat:~/devel/reportlab/REPOS/reportlab/tests $ python39 -X oldparser Python 3.9.0a6 (default, Apr 29 2020, 07:46:29) [GCC 9.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. norm=lambda m: m+(m and(m[-1]!='\n'and'\n'or'')or'\n') File "", line 1 norm=lambda m: m+(m and(m[-1]!='\n'and'\n'or'')or'\n') ^ SyntaxError: invalid string prefix so presumably there has been some parser / language change which renders and'\n' illegal. Is this a real syntax error or an alpha issue? It looks like the tokenization has changed. Putting in the obvious spaces removes the syntax error. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Reduce waiting queue at supermarket from Corona with Python-Webapp
On 16/03/2020 17:38, Orges Leka wrote: This would reduce the waiting queue at the supermarket and possibly the contact to other people thus would help a little bit in slowing down the spread of the Corona virus. Unfortunately, I suspect nothing will alter the number of people in queues in British supermarkets. I am an older person and am probably a bit old-fashioned. I have been shocked at the way people are rushing the shops. Late this week main shops announced first hour would be reserved for us oldies. That hasn't worked with scuffles and the elderly being pushed aside etc etc. Women and children first is long gone the new British attitude is devil take the hindmost :) -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: link to venv python sees a different sys.path
On 12/03/2020 09:19, Dieter Maurer wrote: Let me try again: When you start "python", its "site" module extends "sys.path" to include an appropriate "site-packages". For this, it uses a heuristic based on the path with which you have called the interpreter: i.e. it tries to use dirname()/../lib/python/site-packages If this does not exist, it falls back to the base interpreter' "site-packages". This works fine when you do not use symbolic links. It can work when your symbolic links do not let the heuristic fail. As you have observed, some kind of symbolic links break it, however. Ok I see what you are getting at. However, this sort of defeats the purpose of virtual environments for my case ie when I want many different versions of python all in one place to use for testing. It's fairly clear that links to links to links make it quite hard to decide where the real environment is. However, since the virtual environment does have a lib parallel to the bin it ought not to be impossible. I can continue to use the 'legacy' version of virtualenv, but it seems as though the latest virtualenv (20.0.8) has changed something and its linked python now has the same behaviour as the venv version even when the --copies option is used. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: link to venv python sees a different sys.path
On 11/03/2020 17:24, Dieter Maurer wrote: Robin Becker wrote at 2020-3-11 15:26 +: I'm trying to understand why python 3.8.2 venv behaves differently when it is executed va a link Make the env rptlab@everest:~/code/hg-repos $ python38 -mvenv __py__/382v ... so the linked version of the venv python sees the base python site packages and not the expected venv site-packages. I guess (!) that it is using the path to the interpreter program in order to locate the venv's "site-packages" and falls back to the system's when this is not possible. If my guess is correct, then some links will not work. But, you should be able to use links which find "site-packages" via "/../lib/python/site-packages". Not sure I understand exactly what you are getting at, but a workaround is presumably feasible in any particular script. The documentation says "You don’t specifically need to activate an environment; activation just prepends the virtual environment’s binary directory to your path, so that “python” invokes the virtual environment’s Python interpreter and you can run installed scripts without having to use their full path. However, all scripts installed in a virtual environment should be runnable without activating it, and run with the virtual environment’s Python automatically." I am not running an installed script, but the actual python, however with venv this is just a link to the base python. It seems as though the above doesn't apply to the python itself which seems kind of dumb. A quick check reveals that the documentation later which says "When a virtual environment is active (i.e., the virtual environment’s Python interpreter is running), the attributes sys.prefix and sys.exec_prefix point to the base directory of the virtual environment, whereas sys.base_prefix and sys.base_exec_prefix point to the non-virtual environment Python installation which was used to create the virtual environment. If a virtual environment is not active, then sys.prefix is the same as sys.base_prefix and sys.exec_prefix is the same as sys.base_exec_prefix (they all point to a non-virtual environment Python installation)." is wrong as if you invoke the env/bin/python directly it does change the sys_exec prefix even though the virtual environment is not 'active'. $ __py__/382v/bin/python -c"import sys;print(sys.exec_prefix)" /home/rptlab/code/hg-repos/__py__/382v whereas that's not the case when invoked via a link $ bin/python382v -c"import sys;print(sys.exec_prefix)" /home/rptlab/bin/../LOCAL/382 I guess the problem is that the venv python itself is just a link to the base python whereas the virtualenv pythons are actual copies of base so presumably know where they live. I guess I'll continue to use an older mechanism that works for me. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
link to venv python sees a different sys.path
I'm trying to understand why python 3.8.2 venv behaves differently when it is
executed va a link
Make the env
rptlab@everest:~/code/hg-repos
$ python38 -mvenv __py__/382v
rptlab@everest:~/code/hg-repos
make a link
$ ln -s ../__py__/382v/bin/python bin/python382v
the venv sys.path
rptlab@everest:~/code/hg-repos > $ __py__/382v/bin/python -c"import
sys;print('\n'.join(sys.path))"
/home/rptlab/LOCAL/382/lib/python38.zip
/home/rptlab/LOCAL/382/lib/python3.8
/home/rptlab/LOCAL/382/lib/python3.8/lib-dynload
/home/rptlab/code/hg-repos/__py__/382v/lib/python3.8/site-packages
rptlab@everest:~/code/hg-repos
the linked python sys.path
$ bin/python382v -c"import sys;print('\n'.join(sys.path))"
/home/rptlab/LOCAL/382/lib/python38.zip
/home/rptlab/LOCAL/382/lib/python3.8
/home/rptlab/LOCAL/382/lib/python3.8/lib-dynload
/home/rptlab/LOCAL/382/lib/python3.8/site-packages
rptlab@everest:~/code/hg-repos
$
so the linked version of the venv python sees the base python site packages and
not the expected venv site-packages.
Is there a way to make the link work properly.
This problem doesn't seem to occur with older virtualenv made environments.
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: ubuntu python 2.7 variations
On 29/07/2019 12:33, Chris Angelico wrote:
.
Should I always be using self build python versions?
If you want to maintain your own Python, then by all means, go ahead.
I don't maintain my own Python 2.7, but I have a number of Python 3.x
builds, since Debian Stretch doesn't ship with anything newer than
3.5.
I have all the pythons except 2.7 as user builds
The plus sign does indeed mean that it's modified, but often that just
means they backported some (but not all) of the changes in newer 2.7.x
builds. (Which, at the moment, would only be 2.7.16.)
I have just tried a standard make configure dance with 2.7.16 and that python
seems to behave like the original python 2.7.15rc1 version. So it seems the
current system 2.7.15+ is patched in some way differently.
I can't simply switch pythons as I used the system python to create virtual
environments and all of those need rebuilding.
I can't speak specifically about tokenize, but if you're using it for
anything that isn't actually Python code, you're vulnerable to this
kind of change. I wouldn't normally expect it in a point release,
though.
well preppy is using the ast/tokenizer etc etc to put a lot of python into a
template form with {{ }} expressions etc etc
>
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
ubuntu python 2.7 variations
I help maintain apps on a number of machines and recently a bug surfaced which
affected some of them using the preppy template module; the bug ended up being
the way tokenize worked as in the following
#
from StringIO import StringIO
import tokenize, token
L = []
s='i'
tokenize.tokenize(StringIO(s.strip()).readline,lambda *a: L.append(a))
print('L=%s' % repr(L))
#
Older working machines
Python 2.7.15rc1 (default, Nov 12 2018, 14:31:15)\n[GCC 7.3.0] on linux2
L=[(1, 'i', (1, 0), (1, 1), 'i'), (0, '', (2, 0), (2, 0), '')]
upgraded machines showing a bug
Python 2.7.15+ (default, Nov 27 2018, 23:36:35)\n[GCC 7.3.0] on linux2
L=[(1, 'i', (1, 0), (1, 1), 'i'), (4, '', (1, 1), (1, 2), ''), (0, '', (2, 0),
(2, 0), '')]
The extra token represents a linefeed; however StringIO('i').readline() doesn't
contain a linefeed.
so the problem is that parsing a simple string 'i' results in two different
parses. On my Arch machines running 2.7.16 I see the original parse.
The fix was simple and involved using more robust code.
My question is that since the package builders feel able to modify and affect
behaviour in such a simple case will they do it randomly elsewhere perhaps
leading to other less obvious changes.
Should I always be using self build python versions?
It seems that ubuntu feels able to provide packages which are rc versions or
have a + indicating they're modified. They'll probably argue that this improves
things and I shouldn't be using such low level code ... :(
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: pysftp / paramiko problem
... I tried an experiment with a remote server that I control and pysftp works perfectly there. A difference that I know of is that this server is using ubuntu 18.04 and we don't use passwords, but a private_key. Also this server is using the openssh internal sftp. I believe the failing server is using an earlier version of openssh or OS as it wants to use ssh-dss which is now considered unsafe (I believe). -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: pysftp / paramiko problem
On 13/06/2019 18:23, MRAB wrote:
.
What does:
sftp.normalize('.')
return?
It returns '/'.
sftp.chdir('') and that also fails in paramiko as it seems to use
CMD_REALPATH to do that.
File "tsftp.py", line 7, in main
print(sftp.chdir(''))
File "/home/rptlab/tmp/tpy3/lib/python3.6/site-packages/pysftp/__init__.py",
line 524, in chdir
self._sftp.chdir(remotepath)
File
"/home/rptlab/tmp/tpy3/lib/python3.6/site-packages/paramiko/sftp_client.py",
line 662, in chdir
self._cwd = b(self.normalize(path))
File
"/home/rptlab/tmp/tpy3/lib/python3.6/site-packages/paramiko/sftp_client.py",
line 632, in normalize
t, msg = self._request(CMD_REALPATH, path)
File
"/home/rptlab/tmp/tpy3/lib/python3.6/site-packages/paramiko/sftp_client.py",
line 813, in _request
return self._read_response(num)
File
"/home/rptlab/tmp/tpy3/lib/python3.6/site-packages/paramiko/sftp_client.py",
line 865, in _read_response
self._convert_status(msg)
File
"/home/rptlab/tmp/tpy3/lib/python3.6/site-packages/paramiko/sftp_client.py",
line 894, in _convert_status
raise IOError(errno.ENOENT, text)
FileNotFoundError: [Errno 2] No such file.
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: pysftp / paramiko problem
On 13/06/2019 05:56, dieter wrote:
Robin Becker writes:
On 12/06/2019 05:59, dieter wrote:
Robin Becker writes:
I am trying to convert older code that uses ftplib as the endpoint has switched
to sftp only.
...
Well with real sftp I can cd to that path so if it is a symlink it goes
somewhere.
With pysftp I am unable to chdir or cd into it. With a bit of
difficulty I can use subprocess + sshpass + sftp to do the required
transfer.
Maybe, the problem is the "u" prefix.
Can you try your script with Python 3 or encode the unicode
into a native ``str``?
no same happens in a fresh python 3.6 environment
$ cat - > tsftp.py
def main():
import pysftp
with pysftp.Connection('ftp.remote.com', username='me', password='ucl20
11') as sftp:
print('top level')
print(sftp.listdir())
print(sftp.normalize(''))
if __name__ == '__main__':
main()
(tpy3) rptlab@app1:~/tmp/tpy3
$ python tsftp.py
top level
['']
Traceback (most recent call last):
File "tsftp.py", line 9, in
main()
File "tsftp.py", line 6, in main
print(sftp.normalize(''))
File "/home/rptlab/tmp/tpy3/lib/python3.6/site-packages/pysftp/__init__.py",
line 640, in normalize
return self._sftp.normalize(remotepath)
File
"/home/rptlab/tmp/tpy3/lib/python3.6/site-packages/paramiko/sftp_client.py",
line 632, in normalize
t, msg = self._request(CMD_REALPATH, path)
File
"/home/rptlab/tmp/tpy3/lib/python3.6/site-packages/paramiko/sftp_client.py",
line 813, in _request
return self._read_response(num)
File
"/home/rptlab/tmp/tpy3/lib/python3.6/site-packages/paramiko/sftp_client.py",
line 865, in _read_response
self._convert_status(msg)
File
"/home/rptlab/tmp/tpy3/lib/python3.6/site-packages/paramiko/sftp_client.py",
line 894, in _convert_status
raise IOError(errno.ENOENT, text)
FileNotFoundError: [Errno 2] No such file.
this is what real sftp does
(tpy3) rptlab@app1:~/tmp/tpy3
$ sshpass -p ucl2011 sftp m...@ftp.remote.com
Connected to ftp.remote.com.
sftp> cd
sftp> pwd
Remote working directory: /
sftp> ls
OLD GR
Z.pdf
sftp> ^D
(tpy3) rptlab@app1:~/tmp/tpy3
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: pysftp / paramiko problem
On 12/06/2019 05:59, dieter wrote:
Robin Becker writes:
I am trying to convert older code that uses ftplib as the endpoint has switched
to sftp only.
I am using the pysftp wrapper around paramiko.
The following script fails
def main():
import pysftp
with pysftp.Connection('ftp.remote.com', username='me', password='xx')
as sftp:
print('top level')
print(sftp.listdir())
print(sftp.normalize(u''))
From the "sftp" documentation:
| normalize(self, remotepath)
| Return the expanded path, w.r.t the server, of a given path. This
| can be used to resolve symlinks or determine what the server believes
| to be the :attr:`.pwd`, by passing '.' as remotepath.
This suggests that your observation could be explained
by "u''" being a broken symlink.
Well with real sftp I can cd to that path so if it is a symlink it goes
somewhere.
With pysftp I am unable to chdir or cd into it. With a bit of difficulty I can use subprocess + sshpass + sftp to do the required
transfer.
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
pysftp / paramiko problem
I am trying to convert older code that uses ftplib as the endpoint has switched
to sftp only.
I am using the pysftp wrapper around paramiko.
The following script fails
def main():
import pysftp
with pysftp.Connection('ftp.remote.com', username='me', password='xx')
as sftp:
print('top level')
print(sftp.listdir())
print(sftp.normalize(u''))
print('direct list of ')
print(sftp.listdir(u''))
with sftp.cd(u''):
print(sftp.listdir())
if __name__ == '__main__':
main()
when run the program prints [u''] and then fails at the normalize command.
$ python tsftp.py
top level
[u'']
Traceback (most recent call last):
File "tsftp.py", line 13, in
main()
File "tsftp.py", line 6, in main
print(sftp.normalize(u''))
File "/home/rptlab/devel/env/lib/python2.7/site-packages/pysftp/__init__.py",
line 640, in normalize
return self._sftp.normalize(remotepath)
File
"/home/rptlab/devel/env/lib/python2.7/site-packages/paramiko/sftp_client.py",
line 632, in normalize
t, msg = self._request(CMD_REALPATH, path)
File
"/home/rptlab/devel/env/lib/python2.7/site-packages/paramiko/sftp_client.py",
line 813, in _request
return self._read_response(num)
File
"/home/rptlab/devel/env/lib/python2.7/site-packages/paramiko/sftp_client.py",
line 865, in _read_response
self._convert_status(msg)
File
"/home/rptlab/devel/env/lib/python2.7/site-packages/paramiko/sftp_client.py",
line 894, in _convert_status
raise IOError(errno.ENOENT, text)
IOError: [Errno 2] No such file.
I tried other commands, but it seems any attempt to cd to the directory
fails.
Using sftp in the shell directly I needed to add HostKeyAlgorithms=+ssh-dss for
this host.
Any pointers to what the problem could be?
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: PEP 594 cgi & cgitb removal
... More specifically, with CGI the webserver starts a new process for every single request. That's bad enough for a light C program, but it's certainly not a good idea to start a whole new Python process for every request. At least not for any production website or web service that serves any real amount of traffic. That is, for those who didn't know, the reason why CGI fell out of use quite some time ago. Well I'm afraid I cannot agree with that reasoning. ReportLab used cgi exactly because it starts a new process; most of the reports that are generated take more than a second to generate so the startup time is of less importance. Starting up in a clean state is of importance because the sharing of resources across different reports often leads to buggy report code such as relying on an earlier report to load fonts, define colours etc etc. This could perhaps have been done with multiple interpreters eg mod_python, but I think the isolation there is not perfect especially with C extensions. It might be that the new 'cgi' doesn't use the older api and in that sense we could just use wsgi or whatever the new interface is, but I would still use cgitb to provide nicely formatted traceback html. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
PEP 594 cgi & cgitb removal
In PEP 594 t has been proposed that cgi & cgitb should be removed. I suspect I am not the only person in the world that likes using cgi and cgitb. One of the nice features in cgitb is the ability to get a nice traceback with variable values etc etc etc. I have used the underlying mechanism to produce better traceback information on several occasions and not only in cgi applications. I filed a bug against cgitb in 2004 with (apparently unacceptable patches) that is still unfixed. Although cgi is claimed to be dead it is still one of the easiest ways to get a web service to operate; it is also stateless and robust. If cgi and similar are removed from the stdlib python will become significantly less charged. These batteries are not dead they are pining. There is some discussion on the python-dev list of moving these allegedly dead packages to pypi, but of course that means issues of control, where do the sources reside and other politics. Django has a similar feature to cgitb's output for tracebacks, but is too deeply embedded for use elsewhere; is there anything suitable elsewhere? -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.8 new deprecations
Thanks for letting me know. On Tue, 21 May 2019, 17:04 Inada Naoki, wrote: > I plan to remove int support in Python 3.10. > > If this warning is ignored, the extension module will be broken silently > from 3.10. > It is because C is not typesafe here. > > Regards, > > > 2019年5月22日(水) 0:56 Robin Becker : > >> Marius Gedminas has kindly been doing some work with reportlab and python >> 3.8a1 >> >> He reports this new deprecation warning from a c extension build >> >> DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats >> >> Currently we build the extensions for 2.7 3.4-3.7 Windows 32 & 64 MacOS >> and linux. >> >> His patch is changing several ints to Py_ssize_t after defining >> PY_SSIZE_T_CLEAN. >> >> Can anyone say which versions/runtimes this is needed for or can I just >> assume it has no effect in early versions. >> -- >> Robin Becker >> >> -- >> https://mail.python.org/mailman/listinfo/python-list >> > -- https://mail.python.org/mailman/listinfo/python-list
Python 3.8 new deprecations
Marius Gedminas has kindly been doing some work with reportlab and python 3.8a1 He reports this new deprecation warning from a c extension build DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats Currently we build the extensions for 2.7 3.4-3.7 Windows 32 & 64 MacOS and linux. His patch is changing several ints to Py_ssize_t after defining PY_SSIZE_T_CLEAN. Can anyone say which versions/runtimes this is needed for or can I just assume it has no effect in early versions. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
argparse namespace clashes
After converting a previously working optparse I found that I should not be using 'args' as a destination as it seemed to break django 1.11 management command execution. I changed both the flag name '--args' to '--task-args' and the destination 'args' to 'task-args' and that seemed to fix things. Is there a recommendation anywhere for names and detinations to avoid? -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: using zip for transpose
On 21/02/2019 13:49, Peter Otten wrote: Robin Becker wrote: ... Isn't df.values a numpy array? Then try the more direct and likely more efficient df.values.tolist() or, if you ever want to transpose df.values.T.tolist() The first seems to achieve what your sample code does. (In addition it also converts the numpy type to the corresponding python builtin, i. e. numpy.float64 becomes float etc.) Thanks for the pointer. In fact we were working through all the wrong methods eg iterrows (slow) or iterating over columns which created the need for a transpose. However, df.values.tolist() seems to create a list of row lists which is what is actually needed and it is the fastest. So to convert df to something for reportlab table this seems most efficient rlab_table_data=[['Mean','Max','Min','TestA','TestB']]+df.values.tolist() thanks again -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
using zip for transpose
In conversion of pandas dataframe to reportlab table I suggested using this expression [list(x) for x in map(list,zip(*[df[i].values for i in df]))] which effectively transposes the dataframe. However, it's not clear that this works for a large number of rows. Is the argument *A for A a large list just copied into the zip *args; I suppose calling zip(A[0],A[1],..A[len(A)-1]) cannot be how this is done. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: What is your experience porting Python 2.7.x scripts to Python 3.x?
On 23/01/2019 21:51, Ian Kelly wrote: On Wed, Jan 23, 2019 at 1:36 PM Stefan Behnel wrote: . All right, but apart from absolute imports, the print function, and true division, what has Python 3.x ever done for us? *ducks* headaches :) -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: What is your experience porting Python 2.7.x scripts to Python 3.x?
On 22/01/2019 19:00, Schachner, Joseph wrote: .. For anyone who has moved a substantial bunch of Python 2 to Python 3, can you please reply with your experience? Did you run into any issues? Did 2to3 do its job well, or did you have to review its output to eliminate some working but silly thing? ,.. I did the port of reportlab (www.reportlab.com) from code supporting 2.x only x>=3 to a version which supported 2.7.z & >=3.3. The reportlab toolkit was then about 14 years old and had (and still has lots of cruft). The port effort began 20130215 and ended 20140326 ie 13 months. There were 333 commits on the branch. I used 2to3, but not six. Because we needed to maintain 2.7 and >=3.3 there were quite a few issues related to simple things like iterkeys/values/items <--> keys/values/items removal of xrange etc etc. Maintaining compatible extensions is also hard. Pdf production requires byte output and was already quite messy because reportlab supported both utf8 and unicode inputs; it hasn't got a lot easier. As for performance the 3.x runs were generally slower than 2.7, but I think that situation has changed with 3.6 & 3.7 (for the reportlab tests on windows 2.7 takes 68.7", 3.4 83.8", 3.5 77.0", 3.6 61.5" & 3.7 60.9"). At some point reportlab will be made 3.x only which will require more effort. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: builtins confusion
On 05/11/2018 10:00, Thomas Jollans wrote:
On 2018-11-05 10:47, Robin Becker wrote:
raise ImportError('This package should not be accessible on Python 3. '
'Either you are trying to run from the python-future src
folder '
'or your installation of python-future is corrupted.')
It would appear that this is from the python-future package.
The PyPI page makes the "why?" fairly clear:
https://pypi.org/project/future/
yes I think it comes from pefile which pyinstaller uses. Problem is that this module just hides an import error and may change
other people's attempts at feature recognition.
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
builtins confusion
For some reason I find my windows 32 bit python 2.7.13 has a package called builtins installed in site-packages. This creates
confusion about how to look at builtins.
The __init__.py looks like this
from __future__ import absolute_import
import sys
__future_module__ = True
if sys.version_info[0] < 3:
from __builtin__ import *
# Overwrite any old definitions with the equivalent future.builtins ones:
from future.builtins import *
else:
raise ImportError('This package should not be accessible on Python 3. '
'Either you are trying to run from the python-future src
folder '
'or your installation of python-future is corrupted.')
this creates complete confusion about how to import __builtin__ versus builtins. I think this is associated in some way with
pyinstaller, but am not sure.
Is this a reasonable way to import either builtins / __builtin__
try:
import __builtin__
except ImportError:
import builtins as __builtin__
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: From Mathematica to Jypyter
On 10/10/2018 02:17, Terry Reedy wrote: https://paulromer.net/jupyter-mathematica-and-the-future-of-the-research-paper/ Jupyter, Mathematica, and the Future of the Research Paper Paul Romer, new Nobel prize winner in economics, for research on how ideas interact with economic growth, explained last April why he has switched from Mathematica to Jupyter. I'm a great fan of erroneous spelling and this blog needs a spelling check as this quote shows "Mathematica exemplifies the horde of new Vandals whose pursuit of private gain threatens a far greater pubic loss–the collapse of social systems that took centuries to build." these Vandals are probably not in favour of the #me-too movement either :) -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: This thread is closed [an actual new thread]
On 02/10/2018 11:46, Rhodri James wrote: On 02/10/18 01:02, Ethan Furman wrote: On 10/01/2018 04:26 PM, Ben Finney wrote: > Ethan Furman writes: >> This thread is closed. > > Coming from a moderator of this forum, I don't know how that statement > is to be interpreted. It should be interpreted as: - No further discussion should take place on this thread. - I've done what I can with the primitive tools at hand to block any further discussion. - Continued considerate posts will be discarded. - Continued flame-bait/inconsiderate posts will be met with warnings or stronger as warranted. On what grounds are you suppressing debate? It is exactly and precisely not irrelevant to Python, since it's discussing a Python-specific change to known and understood computing terminology, and frankly the statement "Continued considerate posts will be discarded" is outrageous. I have been unimpressed with the moderation team for some weeks now, but this is just not acceptable. +1 from me there seems to be a considerable humour deficit in some quarters. The language is called python and is named after Monty Python's Flying Circus (https://docs.python.org/2/faq/general.html#why-is-it-called-python). Apparently we now also have the Spanish Inquisition :) -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: [OT] master/slave debate in Python
On 25/09/2018 23:46, Thomas Jollans wrote: .. I have to say I find these unspecified attacks on "SJWs" rather disturbing. Assuming for a moment that "SJW" is a viable insult (that's the way you appear to be using it, though I wouldn't use it myself, in that way or probably at all) - Who is the "SJW brigade" of whom you speak? ... It didn't take me very long to find a connection between this thread and this phrase "I’m Tired of Being Tolerant" on these issues I am with the Voltaireans. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: [OT] master/slave debate in Python
On 24/09/2018 21:40, Kirill Balunov wrote: ... It seems to me that the word "black" has immunity in the next two Python releases ;) So do not worry so much! apparently whitelist/blacklist is an issue so presumably white should also get immunity :) But honestly, it's not pleasant to see how such holy things spread into the world of OSS, and this is apparently only the beginning. +1 -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: [OT] master/slave debate in Python
On 24/09/2018 17:30, Dan Purgert wrote: Robin Becker wrote: [...] just thought control of the wrong sort.. Is there "thought control of the right sort"? yes python is good python is good .... -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: [OT] master/slave debate in Python
On 23/09/2018 15:45, Albert-Jan Roskam wrote: *sigh*. I'm with Hettinger on this. https://www.theregister.co.uk/2018/09/11/python_purges_master_and_slave_in_political_pogrom/ I am as well. Don't fix what is not broken. The semantics (in programming) might not be an exact match, but people have been using these sorts of terms for a long time without anyone objecting. This sort of language control is just thought control of the wrong sort. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On 02/08/2018 08:20, Steven D'Aprano wrote: On Wed, 01 Aug 2018 22:14:54 +0300, Serhiy Storchaka wrote: ... Not always. If your code supported Python 2 in the past, or third-party dependencies supports or supported Python 2, this warning can expose a real bug. Even if all your and third-party code always was Python 3 only, the standard library can contain such kind of bugs. Several years after the EOL of Python 2.7 and moving all living code to Python 3 we can ignore bytes warnings as always false positive. Even then, I don't know that we should do that. I do not believe that the EOL of Python 2 will end all confusion between byte strings and text strings. There is ample opportunity for code to accidentally compare bytes and text even in pure Python 3 code, e.g. comparing data read from files reading from files which are supposed to be opened in the same binary/text mode but aren't. I think I agree; when python 2 is history I can drop all the messing about with bytes in the input and clean up the code a lot. Stuff like svg and xml will still need conversions which are generally unknown in advance. The output will need converting to bytes, but that's fairly standard. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On 01/08/2018 18:19, Peter Otten wrote: I've looked into the actual code which has # paraparser.py f = isPy3 and asBytes or asUnicode K = list(known_entities.keys()) for k in K: known_entities[f(k)] = known_entities[k] It looks like known_entities starts out with the default string type, i. e. unicode in py3 and bytes in py2. While in py2 the code has no effect in py3 it adds the corresponding keys of type bytes. However, known_entities is then used in HTMLParser.handle_entity_ref(self, name) which passes the name as unicode in py3. > I didn't try, but I would suspect that the module keeps working as expected when you remove the lines quoted above. I did try and all the tests pass in 2.7.x & >=3.3; the commit message says "fix entityref handling" and happened during the python 3.x porting. I suppose there was some issue, but its entirely probable that some later change (eg parser) has fixed the original problem and made this code redundant. If I'm correct running the program with the -b flag has at least helped in cleaning up the code in this case. and I will try running all tests under that flag; it's sure to find more issues. In other cases it might detect sources of bugs, so IMHO it's better to have a close look at and possibly rewrite code that triggers the warning rather than to disable it. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On 01/08/2018 14:38, Chris Angelico wrote: t's a warning designed to help people port code from Py2 to Py3. It's not meant to catch every possible comparison. Unless you are actually porting Py2 code and are worried that you'll be accidentally comparing bytes and text, just*don't use the -b switch* and there will be no problems. I don't understand what the issue is here. I don't either, I have never used the -b flag until the issue was raised on bitbucket. If someone is testing a program with reportlab and uses that flag then they get a lot of warnings from this dictionary assignment. Probably the code needs tightening so that we insist on using native strings everywhere; that's quite hard for py2/3 compatible code. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
messing with bytes I discover that this doesn't warn with python -b
if __name__=='__main__':
class nbytes(bytes):
def __eq__(self,other):
return bytes.__eq__(self,other) if isinstance(other,bytes) else
False
def __hash__(self):
return bytes.__hash__(self)
d={'a':1}
d[nbytes(b'a')] = d['a']
print(d)
d={nbytes(b'a'):1}
d['a'] = d[b'a']
print(d)
C:\code\hg-repos\reportlab\tmp>\python37\python -b tb1.py
{'a': 1, b'a': 1}
{b'a': 1, 'a': 1}
I expected one of the assignments to warn.
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
.. Nope, that would be the effect of "and", not "or". "a" is b"b" and "fallback" False "a" is b"b" or "fallback" 'fallback' You seem to be caught in your wrong mental model. I recommend that you let the matter rest for a day or so, and then look at it with a fresh eye. .. my bad; not worrying enough about real problems -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On 01/08/2018 09:52, Chris Angelico wrote:
On Wed, Aug 1, 2018 at 6:36 PM, Robin Becker wrote:
On 31/07/2018 16:52, Chris Angelico wrote:
..
it says explicitly that numeric keys will use numeric comparison, but no
.
Technically, the comparison used is:
a is b or a == b
in other words, identity will match, but mainly, equality is used. The
identity check improves performance in many common cases, and also
avoids pathological cases involving float("nan"), but in general
discussion, it's assumed that value rather than identity is the
comparison used.
ChrisA
If the testing were done in that order then no comparison warning would occur
as inspection shows 'a' is not b'a'
> C:\code\hg-repos\reportlab\tmp>\python37\python.exe -b -Wall
Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
'a' is b'a'
False
so presumably the testing goes something like
a is b or (a==b if comparable(a,b) else False)
even so I still think errors/warnings created internally by implementers should not be exposed. It's not a big deal. I'm a bit
surprised that we don't have a mapping which uses only identity as that would be faster.
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On 31/07/2018 16:52, Chris Angelico wrote: On Wed, Aug 1, 2018 at 1:28 AM, MRAB wrote: On 2018-07-31 08:40, Robin Becker wrote: A bitbucket user complains that python 3.6.6 with -Wall -b prints warnings . The warning looks wrong to be. In Python 2, u'a' and b'a' would be treated as the same key, but in Python 3 they are distinct and can co-exist. Something for Python's bug tracker, I think! It's a warning specifically requested by the -b option. The two keys in question have the same hash, which means they have to be compared directly; they will compare unequal, but because of the -b flag, the comparison triggers a warning. If that warning is spurious, *don't use the -b option*. ChrisA I looked at the language documentation for 3.7 mappings These represent finite sets of objects indexed by nearly arbitrary values. The only types of values not acceptable as keys are values containing lists or dictionaries or other mutable types that are compared by value rather than by object identity, the reason being that the efficient implementation of dictionaries requires a key’s hash value to remain constant. Numeric types used for keys obey the normal rules for numeric comparison: if two numbers compare equal (e.g., 1 and 1.0) then they can be used interchangeably to index the same dictionary entry. it says explicitly that numeric keys will use numeric comparison, but no mention is made of strings/bytes etc etc and there's an implication that object identity is used rather than comparison. In python 3.x b'a' is not the same as 'a' either the documentation is lacking some reference to strings/bytes or the warning is wrong. Using the excuse that normal comparison is being used seems a bit feeble. It would clearly improve speed if object identity were to be used, but I suppose that's not the case for other reasons. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On 31/07/2018 09:16, Paul Moore wrote:
On 31 July 2018 at 08:40, Robin Becker wrote:
A bitbucket user complains that python 3.6.6 with -Wall -b prints warnings
for some reportlab code; the
example boils down to the following
##
C:\code\hg-repos\reportlab\tmp>cat tb.py
if __name__=='__main__':
d={'a':1}
d[b'a'] = d['a']
##
..
v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
b'a' == 'a'
True
b'a' == u'a'
True
which is basically the sort of thing that -b should warn about.
Specifically the quoted code would end up with a dictionary with 2
entries on Python 3, but 1 entry on Python 2.
Paul
yes but I didn't do the compare so this warning seems entirely spurious and wrong. It's not an error to put 1 and 1.0 and 'a' into
a dict. Should I get a warning if the hashes of two different types happen to clash so that an int needs to be checked against a
string?
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Are dicts supposed to raise comparison errors
A bitbucket user complains that python 3.6.6 with -Wall -b prints warnings for
some reportlab code; the
example boils down to the following
##
C:\code\hg-repos\reportlab\tmp>cat tb.py
if __name__=='__main__':
d={'a':1}
d[b'a'] = d['a']
##
C:\code\hg-repos\reportlab\tmp>\python36\python -Wall -b tb.py
tb.py:3: BytesWarning: Comparison between bytes and string
d[b'a'] = d['a']
I had always assumed that dicts didn't care about the type of keys although some types might cause issue with hashability, but
obviously the implementation seems to be comparing b'a' with 'a' (I suppose because they hash to the same chain).
Is this code erroneous or is the warning spurious or wrong?
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: test for absence of infinite loop
On 17/07/2018 12:16, Cameron Simpson wrote: On 17Jul2018 10:10, Robin Becker wrote: A user reported an infinite loop in reportlab. I determined a possible cause and fix and would like to test for absence of the loop. Is there any way to check for presence/absence of an infinite loop in python? I imagine we could do something like call an external process and see if it takes too long, but that seems a bit flaky. While others have kindly pointed you at the halting problem (unsolvable in the general case) you can usually verify that the specific problem you fixed is fixed. Can you figure out how long the task should run with your fix in a test case? Not as time, but in loop iterations? Put a counter in the loop and check that its value doesn't exceed that. well I understand the problem about not halting. However as you point out in a fixed case I know that the test should take fractions of a second to complete. I certainly don't want to put instrumentation into the source code. It's relatively easy to imagine polling termination of a separate thread/process, but that's not particularly reliable. I don't know if there is a way to ask a python interpeter how many instructions it has carried out. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: test for absence of infinite loop
On 17/07/2018 10:32, Chris Angelico wrote: .. All you gotta do is solve the halting problem... https://en.wikipedia.org/wiki/Halting_problem ChrisA ah so it's easy :) -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
test for absence of infinite loop
A user reported an infinite loop in reportlab. I determined a possible cause and fix and would like to test for absence of the loop. Is there any way to check for presence/absence of an infinite loop in python? I imagine we could do something like call an external process and see if it takes too long, but that seems a bit flaky. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: about main()
On 05/07/2018 21:43, Jim Lee wrote: ... identifying the poisonous berries. I would respect your analogy more if every compiler used today were forty years old and not being developed by anyone other than its original creator(s). ChrisA It's not about compilers - it's about skills. As programming becomes more and more specialized, it becomes harder and harder to find programmers with a skill set broad enough to be adaptable to a different task. -Jim I suspect compiler writing is a task that an AI could be taught. The villagers will shout "hey siri I need a compiler" and one will be provided, of course by that time they will all have forgotten about eating berries in favour of soma or equivalent. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.7 configuration issue
On 04/07/2018 05:59, dieter wrote: .. "libffi" contains the so called "foreign function interface" (an interface that allows code written in one language to call code written in another language). Likely, "ctypes" depends on it. Since some time, "ctypes" is part of Python's standard library. It is likely, that the Python test suite contains tests verifying that it could be build successfully. Your observations seem to indicate that "ctypes" contains its own "foreign function interface" specification and that this specification does not fit to the libraries available on your system. The "with-system-ffi" likely tells the Python generation process to use the system "foreign function interface" rather than its own. I don't disagree with the above. I think the issue is that the configure process did not say anything about this. If ctypes is required and cannot be built then ./configure .. should probably tell me; if that's not possible then the make step should do so and fail to complete. Failing at make install seems a bit late. I have to admit that I no longer understand how python builds its extensions; there used to be a step where you configured Modules/Setup*, but that seems no longer required. Any how as an experiment it seems that if I do have the libffi-dev package installed then the with-system-ffi flag is not required and the make install does work. It might be that make silently allows extension builds to fail even if they are 'required' and that later causes an error in the install phase. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Python 3.7 configuration issue
On a Ubuntu trusty system I ran ./configure --prefix=/home/rptlab/PYTHON make && make install and get an error related to the ctypes module not being importable. I needed to do sudo apt-get install libffi-dev ./configure --prefix=/home/rptlab/PYTHON --with-system-ffi make && make install Something in setup.py seems to want ctypes irrespective of what I give to configure; I don't actually know what the alternative to --with-system-ffi does, but it didn't work for me. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Multi-threading with a simple timer?
On 03/07/2018 07:12, Gregory Ewing wrote:
import signal, sys
def timeout(*args):
print("Too late!")
sys.exit(0)
signal.signal(signal.SIGALRM, timeout)
signal.setitimer(signal.ITIMER_REAL, 15)
data = input("Enter something: ")
print("You entered: ", data)
This doesn't work in windows (SIGALRM not available see
https://stackoverflow.com/questions/6947065/right-way-to-run-some-code-with-timeout-in-python)
For completeness I think it needs the handler restoring; this seemed to work for me in linux; I did try using SIG_DFL, but
apparently that just prints 'Alarm clock' and exits at least
import signal, sys, time
def timeout(*args):
print("Too late!")
sys.exit(0)
signal.signal(signal.SIGALRM, timeout)
signal.setitimer(signal.ITIMER_REAL, 15)
data = input("Enter something: ")
signal.signal(signal.SIGALRM,signal.SIG_IGN)
print("You entered: ", data)
for i in reversed(xrange(15)):
print i
time.sleep(1)
print 'finished!'
if I leave out the signal.signal(signal.SIGALRM,signal.SIG_IGN) then the
timeout function gets called anyway.
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
