Re: Trying to use pyinstaller under python 3.11, and, recently started receiving error message about specific module/distribution
Ok, had received response on pyinstaller mailing list, but, also just related to trying clean uninstall/reinstall of modules, but, while checking that out, something else occurred to me, and, it now operates as it should. Anyway, what seemed to be causing issue was actually that, since, while am working under windows 11, quite often you might need to work with case-sensitivity in file names, not by choice, but, since a lot of target platforms, like linux, etc. are case-sensitive, and, at times, when working with external modules, this might cause hassles, etc. In other words, the folder/directory where all my python source code is stored is set to be case-sensitive - there are a couple of ways to implement this under windows 10 and windows 11, via some external utilities, or by running the following command from a terminal/power-shell window, running it as administrator: fsutil.exe file SetCaseSensitiveInfo C:\folder\path enable If you instead use disable argument at the end, it then disables case-sensitivity, and, what did now was, under current project/test code, I created an additional sub-folder, copied code files, etc. over into it, disabled case-sensitivity on it, recreated the virtual environment, and installed all required modules, including pyinstaller using pip, and, when I then run pyinstaller from there, it works fine, and, does what I want it to. In other words, something to do with having case-sensitivity enabled recursively on the folder/directory containing the code and the virtual environment seemed to be causing these errors/issues, specific to altgraph, which doesn't really make sense to me, but, it's now working, so, will archive this to memory, for later reference. Jacob Kruger +2782 413 4791 "Resistance is futile!...Acceptance is versatile..." On 2024/04/02 17:11, Barry wrote: On 1 Apr 2024, at 15:52, Jacob Kruger via Python-list wrote: Found many, many mentions of errors, with some of the same keywords, but, no resolutions that match my exact issue at all. Try asking the pyinstaller developers. I think there is a mailing list. Barry As in, most of them are mentioning older versions of python, and, mainly different platforms - mac and linux, but, various google searches have not mentioned much of using it on windows, and having it just stop working. Now did even try shifting over to python 3.12, but, still no-go. If launch pyinstaller under python 3.10 on this exact same machine, pyinstaller runs - just keep that older version hovering around for a couple of occasional tests, partly since some of my target environments are still running older versions of python, but anyway. Also, not really relevant, but, cx_freeze is perfectly able to generate executables for this same code, but, then not combining all output into a single file - will stick to that for now, but, not always as convenient, and, still wondering what changed here. Jacob Kruger +2782 413 4791 "Resistance is futile!...Acceptance is versatile..." On 2024/03/31 14:51, Barry wrote: On 31 Mar 2024, at 13:24, Jacob Kruger via Python-list wrote: pkg_resources.DistributionNotFound: The 'altgraph' distribution was not found and is required by the application I think I have seen this error being discussed before… A web search for pyinstaller and that error leads to people discussing why it happens it looks like. Barry -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Trying to use pyinstaller under python 3.11, and, recently started receiving error message about specific module/distribution
Ok, last update for now - checked out the following page on pyinstaller.org, and, ended up posting to the mailing list, so, let's see: https://pyinstaller.org/en/latest/when-things-go-wrong.html Jacob Kruger +2782 413 4791 "Resistance is futile!...Acceptance is versatile..." On 2024/04/02 17:11, Barry wrote: On 1 Apr 2024, at 15:52, Jacob Kruger via Python-list wrote: Found many, many mentions of errors, with some of the same keywords, but, no resolutions that match my exact issue at all. Try asking the pyinstaller developers. I think there is a mailing list. Barry As in, most of them are mentioning older versions of python, and, mainly different platforms - mac and linux, but, various google searches have not mentioned much of using it on windows, and having it just stop working. Now did even try shifting over to python 3.12, but, still no-go. If launch pyinstaller under python 3.10 on this exact same machine, pyinstaller runs - just keep that older version hovering around for a couple of occasional tests, partly since some of my target environments are still running older versions of python, but anyway. Also, not really relevant, but, cx_freeze is perfectly able to generate executables for this same code, but, then not combining all output into a single file - will stick to that for now, but, not always as convenient, and, still wondering what changed here. Jacob Kruger +2782 413 4791 "Resistance is futile!...Acceptance is versatile..." On 2024/03/31 14:51, Barry wrote: On 31 Mar 2024, at 13:24, Jacob Kruger via Python-list wrote: pkg_resources.DistributionNotFound: The 'altgraph' distribution was not found and is required by the application I think I have seen this error being discussed before… A web search for pyinstaller and that error leads to people discussing why it happens it looks like. Barry -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Trying to use pyinstaller under python 3.11, and, recently started receiving error message about specific module/distribution
Found many, many mentions of errors, with some of the same keywords, but, no resolutions that match my exact issue at all. As in, most of them are mentioning older versions of python, and, mainly different platforms - mac and linux, but, various google searches have not mentioned much of using it on windows, and having it just stop working. Now did even try shifting over to python 3.12, but, still no-go. If launch pyinstaller under python 3.10 on this exact same machine, pyinstaller runs - just keep that older version hovering around for a couple of occasional tests, partly since some of my target environments are still running older versions of python, but anyway. Also, not really relevant, but, cx_freeze is perfectly able to generate executables for this same code, but, then not combining all output into a single file - will stick to that for now, but, not always as convenient, and, still wondering what changed here. Jacob Kruger +2782 413 4791 "Resistance is futile!...Acceptance is versatile..." On 2024/03/31 14:51, Barry wrote: On 31 Mar 2024, at 13:24, Jacob Kruger via Python-list wrote: pkg_resources.DistributionNotFound: The 'altgraph' distribution was not found and is required by the application I think I have seen this error being discussed before… A web search for pyinstaller and that error leads to people discussing why it happens it looks like. Barry -- https://mail.python.org/mailman/listinfo/python-list
Trying to use pyinstaller under python 3.11, and, recently started receiving error message about specific module/distribution
This started happening this past week, and, while it's worked fine in
the past, the moment I try to launch the pyinstaller process at all, to
generate compiled output, or even if just launch it with no command line
options, I receive the following error message:
pkg_resources.DistributionNotFound: The 'altgraph' distribution was not
found and is required by the application
The full contents of the output string when I even try to just launch
pyinstaller with no commands/arguments is the following:
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File
"C:\pythonScripts\monitoring_nssm\venv\Scripts\pyinstaller.exe\__main__.py",
line 7, in
File
"C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\PyInstaller\__main__.py",
line 228, in _console_script_run
run()
File
"C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\PyInstaller\__main__.py",
line 170, in run
parser = generate_parser()
^
File
"C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\PyInstaller\__main__.py",
line 136, in generate_parser
import PyInstaller.building.build_main
File
"C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\PyInstaller\building\build_main.py",
line 28, in
from PyInstaller.building.api import COLLECT, EXE, MERGE, PYZ
File
"C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\PyInstaller\building\api.py",
line 32, in
from PyInstaller.building.splash import Splash # argument type
validation in EXE
^^
File
"C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\PyInstaller\building\splash.py",
line 23, in
from PyInstaller.depend import bindepend
File
"C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\PyInstaller\depend\bindepend.py",
line 25, in
from PyInstaller.depend import dylib, utils
File
"C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\PyInstaller\depend\utils.py",
line 31, in
from PyInstaller.lib.modulegraph import modulegraph
File
"C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\PyInstaller\lib\modulegraph\modulegraph.py",
line 34, in
from altgraph.ObjectGraph import ObjectGraph
File
"C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\altgraph\__init__.py",
line 144, in
__version__ = pkg_resources.require("altgraph")[0].version
^
File
"C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\pkg_resources\__init__.py",
line 952, in require
needed = self.resolve(parse_requirements(requirements))
^^
File
"C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\pkg_resources\__init__.py",
line 813, in resolve
dist = self._resolve_dist(
^^^
File
"C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\pkg_resources\__init__.py",
line 854, in _resolve_dist
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'altgraph' distribution was not
found and is required by the application
# ---end of output---
I have tried completely removing python's installation, and,
reinstalling it, but, same issue more or less immediately.
If I freeze pip's installed list within this specific virtual
environment, it lists the following:
altgraph==0.17.4
packaging==24.0
pefile==2023.2.7
pyinstaller==6.5.0
pyinstaller-hooks-contrib==2024.3
pywin32-ctypes==0.2.2
# ---end of requirements.txt---
And, if, just for testing, I launch python interpreter, and, ask it to
import altgraph, it provides the same last line of error output?
If relevant, running with python 3.11.8, under windows 11 64-bit, and,
can't think of anything that specifically occurred/changed this past
week, besides normal things like windows updates, etc., but, don't
really think that's likely to be relevant, unless something to do with
pywin32 has caused an issue?
Should I try installing python 3.12 version instead and see if it changes?
Also, if relevant, while running under latest up-to-date version of
windows 11 64 bit, have in any case enabled case-sensitivity on the
folder I store all my python code in, just in case.
TIA
--
Jacob Kruger
+2782 413 4791
"Resistance is futile!...Acceptance is versatile..."
--
https://mail.python.org/mailman/listinfo/python-list
Re: Variable scope inside and outside functions - global statement being overridden by assignation unless preceded by reference
Thanks again, all.
I think the python -i scoping2.py would have given me a good beginning
as well - will archive that one for use.
And, to maybe explain how I work - not an excuse at all - but, I am
actually 100% blind, so a lot of the IDE's, or their common
means/methods of interaction don't suit me all the time, which is why I
generally work via programmer's text editor interfaces, or treat
something like VS code as such, but then still prefer to run my code via
command line, using pdb to then play around with forms of debugging, etc.
And, yes, also generally prefer to work via classes, modules, etc. at
runtime, but this was more or less mostly testing, which then caused
confusion/interference on my side...LOL!
Jacob Kruger
+2782 413 4791
"Resistance is futile!...Acceptance is versatile..."
On 2024/03/07 03:55, Grant Edwards via Python-list wrote:
On 2024-03-07, dn via Python-list wrote:
The idea of importing a module into the REPL and then (repeatedly)
manually entering the code to set-up and execute is unusual (surely type
such into a script (once), and run that (repeatedly). As you say, most
of us would be working from an IDE and hitting 'Run'. Am wondering why
you weren't - but it's not important.
Unless the code is intended to be used as a module, 'import'ing it into
the REPL doesn't make sense.
A simple example:
---testit.py--
x = 'x'
y = 'y'
def foo():
global y
print("hi")
x = 'X'
y = 'Y'
print(x)
print(y)
--
The usual method to play with that interactively is
$ python -i testit.py
>>> x
'x'
>>> y
'y'
>>> foo()
hi
X
Y
>>> x
'x'
>>> y
'Y'
>>>
As we've seen, doing a 'from testit.py import *' doesn't let you test
what the OP was trying to test. Doing 'import testit.py' gets you
closer, but it's a hassle to test code that way. The right thing to do
is 'python -i ' (or the equivalent button/option in an IDE).
https://docs.python.org/3/tutorial/interpreter.html
If you intended to use testit.py as a module, and wanted to experiment
with its behavior as a module, then go ahead and import it. But, don't
do 'from testit.py import *' until
1. you know how that differs from 'import testit.py'
and
2. you want to use that difference
--
https://mail.python.org/mailman/listinfo/python-list
Re: Variable scope inside and outside functions - global statement being overridden by assignation unless preceded by reference
Ok, Ethan, that makes sense - I generally work with modules in folders, etc., but, this was just test code, but, 'see' if I instead import scoping2 as sc2, and then refer to sc2.dt_expiry and sc2.do_it, then it does operate as it should - thanks, again. Jacob Kruger +2782 413 4791 "Resistance is futile!...Acceptance is versatile..." On 2024/03/06 18:57, Ethan Furman via Python-list wrote: On 3/6/24 08:28, Jacob Kruger via Python-list wrote: > C:\temp\py_try>python > Python 3.11.7 (tags/v3.11.7:fa7a6f2, Dec 4 2023, 19:24:49) [MSC v.1937 64 bit (AMD64)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> from scoping2 import * And it becomes clear: only do `from ... import *` when the module has been specifically designed to support that. If you were to also do `import scoping2` and, after calling `do_it()`, `print(scoping2.dt_expiry)`, you would see that it had changed. I know there are good explanations for how variables and names work in Python, but I couldn't find any at the moment. Sorry. -- ~Ethan~ -- https://mail.python.org/mailman/listinfo/python-list
Re: Variable scope inside and outside functions - global statement being overridden by assignation unless preceded by reference
You'll see more details in other mail, but, here I am firing up standard python interpreter from within windows terminal, and then executing following line: from scoping2 import * And, this is under windows 11 windows terminal, which is where I generally interact with my python code, via command line - generally working with flask, and/or other forms of command line interaction, most of the time. Jacob Kruger +2782 413 4791 "Resistance is futile!...Acceptance is versatile..." On 2024/03/06 17:39, Roel Schroeven via Python-list wrote: Op 6/03/2024 om 13:55 schreef Jacob Kruger via Python-list: If you import the contents of that file into the python interpreter, [...] What exactly to you mean by "import the contents of that file into the python interpreter"? Other people have put your code in a script, executed it, and saw it working as expected. I pasted in IPython, and likewise saw it working as expected, and the same with IDLE. It seems to me you must be doing something different from us; maybe the way you execute that code might be the key to this whole confusion. -- https://mail.python.org/mailman/listinfo/python-list
Re: Variable scope inside and outside functions - global statement being overridden by assignation unless preceded by reference
Matt, other mail is more relevant - seems to maybe have more to do with
different behavour if import code, or not - no, does not make sense to
me - but, here's the command line contents including printing out id()
results, but, only working via importing code:
#---start session---
C:\temp\py_try>type scoping2.py
from datetime import datetime, timezone, timedelta
dt_expiry = datetime.strptime("1970-01-01 00:00", "%Y-%m-%d
%H:%M").replace(tzinfo=timezone.utc)
def do_it():
global dt_expiry
dt_expiry = datetime.now()+timedelta(minutes=5)
print("date value", dt_expiry.strftime("%Y-%m-%d %H:%M"))
print("ID", id(dt_expiry))
# end of do_it function
C:\temp\py_try>python
Python 3.11.7 (tags/v3.11.7:fa7a6f2, Dec 4 2023, 19:24:49) [MSC v.1937
64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from scoping2 import *
>>> print(dt_expiry)
1970-01-01 00:00:00+00:00
>>> print(id(dt_expiry))
1808577867152
>>> do_it()
date value 2024-03-06 18:39
ID 1808572660736
>>> print(dt_expiry)
1970-01-01 00:00:00+00:00
>>> print(id(dt_expiry))
1808577867152
>>>
---end session---
As in, the two different ID values are being returned outside and inside
the function, whereas, if I included that bit inside the interpreter
while typing code manually, chances are the same ID would be retained
both inside and outside function.
Jacob Kruger
+2782 413 4791
"Resistance is futile!...Acceptance is versatile..."
On 2024/03/06 15:57, Mats Wichmann via Python-list wrote:
On 3/6/24 05:55, Jacob Kruger via Python-list wrote:
Ok, simpler version - all the code in a simpler test file, and
working with two separate variables to explain exactly what am
talking about:
If you import the contents of that file into the python interpreter,
dt_expiry will start off as "1970-01-01 00:00", and, if you execute
do_it function, it will print out the new value assigned to the
dt_expiry variable inside that function, but if you then again check
the value of the dt_expiry variable afterwards, it's reverted to the
1970... value?
If I take out the line that removes values from l_test #
l_test.clear() # before appending new value to it, then it will also
not retain it's new/additional child items after the function exits,
and will just revert back to [1, 2, 3] each and every time.
In other words, with some of the variable/object types, if you use a
function that manipulates the contents of a variable, before then
re-assigning it a new value, it seems like it might then actually
update/manipulate the global variable, but, either just calling
purely content retrieval functions against said objects, or assigning
them new values from scratch seems to then ignore the global scope
specified in the first line inside the function?
Hope this makes more sense
No, it doesn't. Your code is working as one would expect. For example,
adding prints for the l_test variable, and removing the .clear() which
you claim makes it not work, shows me:
before: l_test=[1, 2, 3], id(l_test)=140153285385856
leaving do_it: l_test=[1, 2, 3, 1, 2, 3, 99], id(l_test)=140153285385856
after: l_test=[1, 2, 3, 1, 2, 3, 99], id(l_test)=140153285385856
It's the same list object, as you can see by the id values. And the
list is updating as expected.
And... you don't need the global statement for l_test. As it's
mutable, you can mutate it in the function; the global only acts on
assignment. Using "global" for that may make your intent more clear to
readers though, although static checkers will grumble at you.
You must be doing something additional that you're not telling us about.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Variable scope inside and outside functions - global statement being overridden by assignation unless preceded by reference
Thanks for all your input people, and, yes, I know that besides the
scope oddities the rest of the code is not my normal style either - was
partly due to forms of experimentation to try figure out what could be
causing issues. For example, instead of [:] syntax, was specifically
using copy() to make sure was not interacting with original variable
values, etc.
This will be a bit longer - copying-pasting command line output here to
show you what I truly mean - first session, where I am importing code
into interpreter and second session where I retype exact same code
behave differently:
#---first session---
C:\temp\py_try>type scoping2.py
from datetime import datetime, timezone, timedelta
dt_expiry = datetime.strptime("1970-01-01 00:00", "%Y-%m-%d
%H:%M").replace(tzinfo=timezone.utc)
def do_it():
global dt_expiry
dt_expiry = datetime.now()+timedelta(minutes=5)
print(dt_expiry.strftime("%Y-%m-%d %H:%M"))
# end of do_it function
C:\temp\py_try>python
Python 3.11.7 (tags/v3.11.7:fa7a6f2, Dec 4 2023, 19:24:49) [MSC v.1937
64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from scoping2 import *
>>> print(dt_expiry)
1970-01-01 00:00:00+00:00
>>> do_it()
2024-03-06 18:12
>>> print(dt_expiry)
1970-01-01 00:00:00+00:00
>>>
#---end first session---
And, if I now retype the contents of the file into the python
interpreter instead:
#---start second session---
C:\temp\py_try>python
Python 3.11.7 (tags/v3.11.7:fa7a6f2, Dec 4 2023, 19:24:49) [MSC v.1937
64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from datetime import datetime, timezone, timedelta
>>> dt_expiry = datetime.strptime("1970-01-01 00:00", "%Y-%m-%d
%H:%M").replace(tzinfo=timezone.utc)
>>> def do_it():
... global dt_expiry
... dt_expiry = datetime.now()+timedelta(minutes=5)
... print(dt_expiry.strftime("%Y-%m-%d %H:%M"))
...
>>> print(dt_expiry)
1970-01-01 00:00:00+00:00
>>> do_it()
2024-03-06 18:20
>>> print(dt_expiry)
2024-03-06 18:20:03.909825
>>>
#---end second session---
So, in the second session, where retyped everything, it behaves as I
would expect it to, but, during first session, the variable is being
treated as a local variable inside the function - no code differences
since literally copied-pasted each and every line into console, but, a
different behaviour nonetheless?
So, yes, know this comes across like some form of a scam/joke, or
list-garbage, since it doesn't make any sense to me at all, but still
just wondering if missing something, or should I shift over to 3.12 to
see if if works differently, or just try reinstalling 3.11 from scratch,
or should I retry the above in something like the VS code console, or a
different python console, etc.?
Sorry
Jacob Kruger
Jacob Kruger
+2782 413 4791
"Resistance is futile!...Acceptance is versatile..."
On 2024/03/06 16:01, Thomas Passin via Python-list wrote:
On 3/6/2024 7:55 AM, Jacob Kruger via Python-list wrote:
Ok, simpler version - all the code in a simpler test file, and
working with two separate variables to explain exactly what am
talking about:
# start code
from datetime import datetime, timezone, timedelta
from copy import copy
# initialise original values
dt_expiry = datetime.strptime("1970-01-01 00:00", "%Y-%m-%d
%H:%M").replace(tzinfo=timezone.utc)
l_test = [1, 2, 3]
def do_it():
global dt_expiry, l_test # asked python to refer to global
variables for both
# assign new value immediately
dt_expiry = datetime.now()+timedelta(minutes=5)
print(dt_expiry.strftime("%Y-%m-%d %H:%M")) # just to show new
value has been assigned
# grab copy of list for re-use of items
l_temp = copy(l_test)
# following line means l_test will later on retain value in
global scope because it was manipulated inside function instead of
just assigned new value
l_test.clear()
# replace original set of values
for i in l_temp: l_test.append(i)
# add new item
l_test.append(99)
# end of do_it function
# end code
If you import the contents of that file into the python interpreter,
dt_expiry will start off as "1970-01-01 00:00", and, if you execute
do_it function, it will print out the new value assigned to the
dt_expiry variable inside that function, but if you then again check
the value of the dt_expiry variable afterwards, it's reverted to the
1970... value?
Not when I run your code. With a little annotation added to the print
statements I get (I added the import statements to make it run, and I
used the same date-time formatting for all three pr
Re: Variable scope inside and outside functions - global statement being overridden by assignation unless preceded by reference
So, this does not make sense to me in terms of the following snippet
from the official python docs page:
https://docs.python.org/3/faq/programming.html
"In Python, variables that are only referenced inside a function are
implicitly global. If a variable is assigned a value anywhere within the
function’s body, it’s assumed to be a local unless explicitly declared
as global."
So, I would then assume that if I explicitly include a variable name
inside the global statement, then even just assigning it a new value
should update the variable in the global context, outside the function?
Unless this is something that changed from 3.11 to 3.12 - since that
snippet is more or less referring to 3.12, but, don't think it was
changed in any way?
Jacob Kruger
+2782 413 4791
"Resistance is futile!...Acceptance is versatile..."
On 2024/03/06 14:55, Jacob Kruger wrote:
Ok, simpler version - all the code in a simpler test file, and working
with two separate variables to explain exactly what am talking about:
# start code
from datetime import datetime, timezone, timedelta
from copy import copy
# initialise original values
dt_expiry = datetime.strptime("1970-01-01 00:00", "%Y-%m-%d
%H:%M").replace(tzinfo=timezone.utc)
l_test = [1, 2, 3]
def do_it():
global dt_expiry, l_test # asked python to refer to global
variables for both
# assign new value immediately
dt_expiry = datetime.now()+timedelta(minutes=5)
print(dt_expiry.strftime("%Y-%m-%d %H:%M")) # just to show new
value has been assigned
# grab copy of list for re-use of items
l_temp = copy(l_test)
# following line means l_test will later on retain value in global
scope because it was manipulated inside function instead of just
assigned new value
l_test.clear()
# replace original set of values
for i in l_temp: l_test.append(i)
# add new item
l_test.append(99)
# end of do_it function
# end code
If you import the contents of that file into the python interpreter,
dt_expiry will start off as "1970-01-01 00:00", and, if you execute
do_it function, it will print out the new value assigned to the
dt_expiry variable inside that function, but if you then again check
the value of the dt_expiry variable afterwards, it's reverted to the
1970... value?
If I take out the line that removes values from l_test #
l_test.clear() # before appending new value to it, then it will also
not retain it's new/additional child items after the function exits,
and will just revert back to [1, 2, 3] each and every time.
In other words, with some of the variable/object types, if you use a
function that manipulates the contents of a variable, before then
re-assigning it a new value, it seems like it might then actually
update/manipulate the global variable, but, either just calling purely
content retrieval functions against said objects, or assigning them
new values from scratch seems to then ignore the global scope
specified in the first line inside the function?
Hope this makes more sense
Jacob Kruger
+2782 413 4791
"Resistance is futile!...Acceptance is versatile..."
On 2024/03/05 20:23, dn via Python-list wrote:
Jacob,
Please reduce the problem to a small code-set which reproduces the
problem. If we can reproduce same, then that tells us something. At
the very least, we can experiment without having to expend amounts of
time in a (likely faulty) bid to reproduce the same environment.
Also, code is the ultimate description!
Perhaps start with a small experiment:
- after l_servers is created, print its id()
- after the global statement, print its id()
- after the clear/reassignment, print its id()
Is Python always working with the same list?
Please advise...
On 6/03/24 07:13, Jacob Kruger via Python-list wrote:
Hi there
Working with python 3.11, and, issue that confused me for a little
while, trying to figure out what was occurring - unless am
completely confused, or missing something - was that, for example,
when having pre-defined a variable, and then included it in the
global statement inside a function, that function was still
referring to a completely local instance, without manipulating
outside variable object at all unless I first executed a form of
referral to it, before then possibly assigning a new value to it.
Now, this does not seem to occur consistently if, for example, I
just run bare-bones test code inside the python interpreter, but
consistently occurs inside my actual testing script.
Basically, in a file with python code in that am using for a form of
testing at the moment, at the top of the file, under all the import
statements, I initiate the existence of a list variable to make use of
later:
# code snippet
l_servers = []
# end of first code snippet
Then, lower down, inside a couple of different functions, the first
line
inside the functions includes
Re: Variable scope inside and outside functions - global statement being overridden by assignation unless preceded by reference
Ok, simpler version - all the code in a simpler test file, and working
with two separate variables to explain exactly what am talking about:
# start code
from datetime import datetime, timezone, timedelta
from copy import copy
# initialise original values
dt_expiry = datetime.strptime("1970-01-01 00:00", "%Y-%m-%d
%H:%M").replace(tzinfo=timezone.utc)
l_test = [1, 2, 3]
def do_it():
global dt_expiry, l_test # asked python to refer to global
variables for both
# assign new value immediately
dt_expiry = datetime.now()+timedelta(minutes=5)
print(dt_expiry.strftime("%Y-%m-%d %H:%M")) # just to show new
value has been assigned
# grab copy of list for re-use of items
l_temp = copy(l_test)
# following line means l_test will later on retain value in global
scope because it was manipulated inside function instead of just
assigned new value
l_test.clear()
# replace original set of values
for i in l_temp: l_test.append(i)
# add new item
l_test.append(99)
# end of do_it function
# end code
If you import the contents of that file into the python interpreter,
dt_expiry will start off as "1970-01-01 00:00", and, if you execute
do_it function, it will print out the new value assigned to the
dt_expiry variable inside that function, but if you then again check the
value of the dt_expiry variable afterwards, it's reverted to the 1970...
value?
If I take out the line that removes values from l_test # l_test.clear()
# before appending new value to it, then it will also not retain it's
new/additional child items after the function exits, and will just
revert back to [1, 2, 3] each and every time.
In other words, with some of the variable/object types, if you use a
function that manipulates the contents of a variable, before then
re-assigning it a new value, it seems like it might then actually
update/manipulate the global variable, but, either just calling purely
content retrieval functions against said objects, or assigning them new
values from scratch seems to then ignore the global scope specified in
the first line inside the function?
Hope this makes more sense
Jacob Kruger
+2782 413 4791
"Resistance is futile!...Acceptance is versatile..."
On 2024/03/05 20:23, dn via Python-list wrote:
Jacob,
Please reduce the problem to a small code-set which reproduces the
problem. If we can reproduce same, then that tells us something. At
the very least, we can experiment without having to expend amounts of
time in a (likely faulty) bid to reproduce the same environment.
Also, code is the ultimate description!
Perhaps start with a small experiment:
- after l_servers is created, print its id()
- after the global statement, print its id()
- after the clear/reassignment, print its id()
Is Python always working with the same list?
Please advise...
On 6/03/24 07:13, Jacob Kruger via Python-list wrote:
Hi there
Working with python 3.11, and, issue that confused me for a little
while, trying to figure out what was occurring - unless am completely
confused, or missing something - was that, for example, when having
pre-defined a variable, and then included it in the global statement
inside a function, that function was still referring to a completely
local instance, without manipulating outside variable object at all
unless I first executed a form of referral to it, before then
possibly assigning a new value to it.
Now, this does not seem to occur consistently if, for example, I just
run bare-bones test code inside the python interpreter, but
consistently occurs inside my actual testing script.
Basically, in a file with python code in that am using for a form of
testing at the moment, at the top of the file, under all the import
statements, I initiate the existence of a list variable to make use of
later:
# code snippet
l_servers = []
# end of first code snippet
Then, lower down, inside a couple of different functions, the first line
inside the functions includes the following:
# code snippet
global l_servers
# end code snippet
That should, in theory, mean that if I assign a value to that variable
inside one of the functions, it should reflect globally?
However, it seems like that, while inside those functions, it can be
assigned a new list of values, but if I then return to the scope outside
the functions, it has reverted back to being an empty list = []?
The issue seems to specifically (or not) occur when I make a call to
one function, and, in the steps it's executing in one context, while
it's not doing anything to the list directly, it's then making a call
to the second function, which is then meant to repopulate the list
with a brand new set of values.
Now, what almost seems to be occurring, is that while just
manipulating the contents of a referenced variable is fine in this
context, the moment I try to reassign it, that's where the i
Variable scope inside and outside functions - global statement being overridden by assignation unless preceded by reference
Hi there Working with python 3.11, and, issue that confused me for a little while, trying to figure out what was occurring - unless am completely confused, or missing something - was that, for example, when having pre-defined a variable, and then included it in the global statement inside a function, that function was still referring to a completely local instance, without manipulating outside variable object at all unless I first executed a form of referral to it, before then possibly assigning a new value to it. Now, this does not seem to occur consistently if, for example, I just run bare-bones test code inside the python interpreter, but consistently occurs inside my actual testing script. Basically, in a file with python code in that am using for a form of testing at the moment, at the top of the file, under all the import statements, I initiate the existence of a list variable to make use of later: # code snippet l_servers = [] # end of first code snippet Then, lower down, inside a couple of different functions, the first line inside the functions includes the following: # code snippet global l_servers # end code snippet That should, in theory, mean that if I assign a value to that variable inside one of the functions, it should reflect globally? However, it seems like that, while inside those functions, it can be assigned a new list of values, but if I then return to the scope outside the functions, it has reverted back to being an empty list = []? The issue seems to specifically (or not) occur when I make a call to one function, and, in the steps it's executing in one context, while it's not doing anything to the list directly, it's then making a call to the second function, which is then meant to repopulate the list with a brand new set of values. Now, what almost seems to be occurring, is that while just manipulating the contents of a referenced variable is fine in this context, the moment I try to reassign it, that's where the issue is occurring . Here are relevant excerpts from the file:- # start code # original assignation in main part of file l_servers = [] # function wich is initially being executed def interact(): global l_servers # extra code inbetween choosing what to carry out # ... # end of other code bl_response, o_out = list_servers() if bl_response: # just make sure other function call was successful l_servers.clear() # first make reference to global variable for srv in o_out: l_servers.append(srv) # now re-populate items # end code snippet from inside interact function # end of interact function # end of code snippet That other function being called from within, list_servers() was initially just trying to populate the values inside the global list variable itself, but was ending up in a similar fashion - reverting to initial empty value, but, the above now seems to work, as long as I first make reference to/manipulate/work with global variable instead of just trying to reassign it a brand new value/set of items? So, am I missing something obvious, have I forgotten about something else - yes, know that if was working from within an embedded function, I might need/want to then use the nonlocal statement against that variable name, but, honestly, just not sure how this can be occurring, and, it's not just with this one list variable, etc.? If I try simple test code from within the python interpreter, using different types of variables, this does also not seem to be the same all the time, but, don't think it can relate to an iterable like a list, or else, just in case, here is the code snippet with all the import statements from the top of that file, in case something could be overriding standard behaviour - not likely in this context, but, really not sure what's occurring: # import code snippet import requests, time from requests.auth import HTTPBasicAuth import psutil as psu import pytz import bcrypt from copy import copy from datetime import datetime, timedelta, timezone from dateutil.parser import parse # end of import snippet Thanks if you have any ideas/thoughts on the matter Jacob Kruger +2782 413 4791 "Resistance is futile!...Acceptance is versatile..." -- https://mail.python.org/mailman/listinfo/python-list
Re: No current way to just compile flet code into truly native packages for smart phones, etc.?
Yup. Also checked out beeware - which also offers cross-platform compilation of same code - but, one minor issue there first time tried it out was working with additional/external modules, and, packaging resources. Plus, while it includes it's own form of LBC GUI interface - toga - the one included here in flet seemed 'nicer': https://beeware.org/project/projects/libraries/toga/ But, haven't taken time to really take it much further than just a bit of playing around so far. Jacob Kruger +2782 413 4791 "Resistance is futile!...Acceptance is versatile..." On 2023/11/13 19:47, Barry wrote: On 13 Nov 2023, at 17:21, Jacob Kruger via Python-list wrote: Had a look at the following bit of introduction to using python and flet to build cross-platform flutter-based apps using same python code, and, while it seems to work alright if tell it to run as under GUI here on windows desktop, and, while can get it to fire up PWA version as well, that's not really stand-alone since will still require code to be running in background, and, in terms of, for example, android, it seems like it will need to then be running via the native android flet interpreter for it to work as such? https://flet.dev/docs/ Flet PWA deployment <https://flet.dev/docs/guides/python/deploying-web-app/progressive-web-apps/> flet looks interesting. It seems from the road map there is lots missing today. Also the people that know flet are on discord. You may find someone here. Barry -- Jacob Kruger +2782 413 4791 "Resistance is futile!...Acceptance is versatile..." -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
No current way to just compile flet code into truly native packages for smart phones, etc.?
Had a look at the following bit of introduction to using python and flet to build cross-platform flutter-based apps using same python code, and, while it seems to work alright if tell it to run as under GUI here on windows desktop, and, while can get it to fire up PWA version as well, that's not really stand-alone since will still require code to be running in background, and, in terms of, for example, android, it seems like it will need to then be running via the native android flet interpreter for it to work as such? https://flet.dev/docs/ Flet PWA deployment <https://flet.dev/docs/guides/python/deploying-web-app/progressive-web-apps/> -- Jacob Kruger +2782 413 4791 "Resistance is futile!...Acceptance is versatile..." -- https://mail.python.org/mailman/listinfo/python-list
Re: SQL rollback of multiple inserts involving constraints
Think performing a session/transaction flush after the first two inserts should offer the workaround before you've committed all transaction actions to the database finally: https://medium.com/@oba2311/sqlalchemy-whats-the-difference-between-a-flush-and-commit-baec6c2410a9 HTH Jacob Kruger +2782 413 4791 "Resistance is futile!...Acceptance is versatile..." On 2023/11/10 11:15, Loris Bennett via Python-list wrote: Hi, In my MariaDB database I have a table 'people' with 'uid' as the primary key and a table 'groups' with 'gid' as the primary key. I have a third table 'memberships' with 'uid' and 'gid' being the primary key and the constraint that values for 'uid' and 'gid' exist in the tables 'people' and 'groups', respectively. I am using SQLAlchemy and writing a method to setup a membership for a new person in a new group. I had assumed that I should be able to perform all three inserts (person, group, membership) with a single transaction and then rollback if there is a problem. However, the problem is that if the both the insert into 'people' and that into 'groups' are not first committed, the constraint on the insertion of the membership fails. What am I doing wrong? Apologies if this is actually an SQL question rather than something related to SQLAlchemy. Cheers, Loris -- https://mail.python.org/mailman/listinfo/python-list
Re: Android APK
Beeware is the only one I have tried out: https://beeware.org/ Jacob Kruger +2782 413 4791 "Resistance is futile...but, acceptance is versatile..." On 2023/01/25 22:55, Jules Tillinghast wrote: Is there a good python library for converting python3 to android APK -- https://mail.python.org/mailman/listinfo/python-list
Re: Recommendations in terms of threading, multi-threading and/or asynchronous processes/programming? - Sent Mail - Mozilla Thunderbird
Ok, the specific usage case right now is that I need to set up a process pulling contents of e-mail messages from an IMAP protocol mail server, which I then populate into a postgresql database, and, since this is the inbox of a relatively large-scale CRM/support system, there are currently over 2.5 million e-mails in the inbox, but, it can grow by over 5 per day. I already have the basic process operating, using imap_tools, but, wanted to enable you to query the process during run-time, without needing to either check logs, or query the database itself while it is on-the-go - even if this is just for initial population time-period, since later on I will just set up code to run under a form of cron job, or handling time-based repeats itself on a separate machine. Also wanted to offer the ability to either pause, or terminate processes while it's busy batch processing large chunks of e-mail messages - either send a message to the thread, or set a global variable to tell it to end the run after the current process item has finished off, just in case. So, I think that for now, threading is probably the simplest to look into. Later on, was also considering forms of low-level monitoring for UI elements, but, this is not really related to initial task, but, could almost relate to forms of non-visual gaming interfaces, for blind/VI individuals - I am myself 100% blind, but, that's not really relevant in this context. Stay well Jacob Kruger +2782 413 4791 "Resistance is futile...but, acceptance is versatile..." On 2023/01/06 21:19, Chris Angelico wrote: On Sat, 7 Jan 2023 at 04:54, jacob kruger wrote: I am just trying to make up my mind with regards to what I should look into working with/making use of in terms of what have put in subject line? As in, if want to be able to trigger multiple/various threads/processes to run in the background, possibly monitoring their states, either via interface, or via global variables, but, possibly while processing other forms of user interaction via the normal/main process, what would be recommended? Any. All. Whatever suits your purpose. They all have different goals, different tradeoffs. Threads are great for I/O bound operations; they're easy to work with (especially in Python), behave pretty much like just having multiple things running concurrently, and generally are the easiest to use. But you'll run into limits as your thread count climbs (with a simple test, I started seeing delays at about 10,000 threads, with more serious problems at 100,000), so it's not well-suited for huge scaling. Also, only one thread at a time can run Python code, which limits them to I/O-bound tasks like networking. Multiple processes take a lot more management. You have to carefully define your communication channels (for instance, a multiprocessing.Queue() to collect results), but they can do CPU-bound tasks in parallel. So multiprocessing is a good way to saturate all of your CPU cores. Big downsides include it being much harder to share information between the processes, and much MUCH higher resource usage than threads (with the same test as the above, I ran into limitations at just over 500 processes - way fewer than the 10,000 threads!). Asynchronous I/O runs a single thread in a single process. So like multithreading, it's only good for I/O bound tasks like networking. It's harder to work with, though, since you have to be very careful to include proper await points, and you can stall out the entire event loop with one mistake (common culprits being synchronous disk I/O, and gethostbyname). But the upside is that you get near-infinite tasks, basically just limited by available memory (or other resources). Use whichever one is right for your needs. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Recommendations in terms of threading, multi-threading and/or asynchronous processes/programming? - Sent Mail - Mozilla Thunderbird
I am just trying to make up my mind with regards to what I should look into working with/making use of in terms of what have put in subject line? As in, if want to be able to trigger multiple/various threads/processes to run in the background, possibly monitoring their states, either via interface, or via global variables, but, possibly while processing other forms of user interaction via the normal/main process, what would be recommended? As in, for example, the following page mentions some possibilities, like threading, asyncio, etc., but, without going into too much detail: https://itnext.io/practical-guide-to-async-threading-multiprocessing-958e57d7bbb8 And, have played around with threading in the past, and, was looking into asyncio now, but, thought would rather first ask for recommendations/suggestions here? For reference, am currently working with python 3.11, or might roll back to 3.10 if relevant, but, main thing is just want to get an idea of what's simplest to make use of in this context? Thanks in advance -- Jacob Kruger +2782 413 4791 "Resistance is futile...but, acceptance is versatile..." -- https://mail.python.org/mailman/listinfo/python-list
Initial introduction
Hi there I might shortly be posting about a specific issue with imaplib/poplib, but, thought would first introduce myself. I am 100% blind, but, among other forms of assistive technology software, I use windows PC's with the NVDA screenreader, which itself is written in python, but, besides that, i generally work with either VS code or other programmer-specific text editors when it comes to python coding, etc., and, while have been a web application developer for over 25 years, working with different programming/scripting languages and platforms, I am more-or-less a full-time python developer nowadays, and, when it comes to web development in this context, I work with either flask or django, but anyway. Regards -- Jacob Kruger +2782 413 4791 "Resistance is futile...but, acceptance is versatile..." -- https://mail.python.org/mailman/listinfo/python-list
Re: New to programming and asking about accessibility with jaws screen reader.
Presume you're talking about python idle IDE? Forget it, and, like said, I use edSharp programmers text editor, and work from command line/console when want to test/run my code. Stay well Jacob Kruger Blind Biker Skype: BlindZA "Resistance is futile, but, acceptance is versatile..." On 2016-07-25 12:14 AM, josphine said wrote: Thanks for your care, Downloaded it from the python download page, (86-64 bit). It reads only the menus, when i try to write anything , readable only with the curser,,. Maybe i don't know exactly how to use it, So if possible that anyone walk with me step by step?? Sorry for that. Thanks, god bless you all. Sent from my iPhone On Jul 24, 2016, at 11:52 PM, Jacob Kruger <ja...@blindza.co.za> wrote: What part is not accessible? Ask since while haven't really gone past 3.4 and 3.5.1, at times, I generally work with specific text/code editors, and the command line window, and this works fine for me with both NVDA, and jaws 17 at times. Stay well Jacob Kruger Blind Biker Skype: BlindZA "Resistance is futile, but, acceptance is versatile..." On 2016-07-24 10:04 PM, josphine said wrote: Hello, I have downloaded python3.6 and found it is not accessible with jaws 17. So, is there any skripts for jaws for that? Or any suggestions? Every guidance will highly appreciated. Thanks. -- https://mail.python.org/mailman/listinfo/python-list
Re: New to programming and asking about accessibility with jaws screen reader.
What part is not accessible? Ask since while haven't really gone past 3.4 and 3.5.1, at times, I generally work with specific text/code editors, and the command line window, and this works fine for me with both NVDA, and jaws 17 at times. Stay well Jacob Kruger Blind Biker Skype: BlindZA "Resistance is futile, but, acceptance is versatile..." On 2016-07-24 10:04 PM, josphine said wrote: Hello, I have downloaded python3.6 and found it is not accessible with jaws 17. So, is there any skripts for jaws for that? Or any suggestions? Every guidance will highly appreciated. Thanks. -- https://mail.python.org/mailman/listinfo/python-list
Re: Help
On 2016-02-28 9:58 PM, tomwilliamson...@gmail.com wrote:
I need to write a program that allows the user to enter a sentence then asks
them which word they wish to find- and then tell them the position that word
was within the sentence.
E.g. Please help with this code
Then they enter help it would return that it appears in the 2nd position in the
sentence.
From what I gather it appears to be a list function but I am struggling to come
up with a solution.
Thanks.
Something along lines of loading the sentence into a string, using the
str.split() function to split it into a list object, then cycling
through to strip all the spaces and/or punctuation out of the
elements/items, and then the list.index() function can return the index
of a word in the list, which is the zero-based position of the item in
that list.
#something along lines of
s = input("enter sentence")
s.replace(",", "") #do this for all common punctuation characters
l = s.split(" ") #space is actually default
s2 = input("enter word")
i = l.index(s2)
print("Your word is at position " + str(i+1) + " in the sentence")
#end code
That's just typed here in message - HTH
HTH
Jacob Kruger
Blind Biker
Skype: BlindZA
"Roger Wilco wants to welcome you...to the space janitor's closet..."
--
https://mail.python.org/mailman/listinfo/python-list
Re: Installing pygame
I literally just installed pyGame under 3.5.1, using following .whl file that pulled off a site offering collections of .whl files: http://www.lfd.uci.edu/~gohlke/pythonlibs/ And, according to following page, the command of pi3p install ...followed by name of .whl file... handled installing pyGame under 3.5.1: https://skellykiernan.wordpress.com/2015/01/04/python-pygame-install/ That was after copying the .whl file into the./scripts directory under python 3.5.1 installation path. HTH Jacob Kruger Blind Biker Skype: BlindZA "Roger Wilco wants to welcome you...to the space janitor's closet..." On 2016-01-21 12:10 PM, John Mycroft wrote: Hi! I have now spent several hours trying to install Pygame with Python 3.5. I have installed from a msi file "successfully" but "import pygame" fails either because Python can't find pygame or because "%1 is not a valid .DLL". I have followed the instructions at https://www.webucator.com/blog/2015/03/installing-the-windows-64-bit-version-of-pygame/ to install from a wheel which works just fine until I get to *** C:\Python>c:\python\scripts\pip install pygame-1.9.2a0-cp35-none-win32 c:\python\lib\site-packages\pip\pep425tags.py:89: RuntimeWarning: Config variable 'Py_DEBUG' is unset, Python ABI tag may be incorrect warn=(impl == 'cp')): c:\python\lib\site-packages\pip\pep425tags.py:93: RuntimeWarning: Config variable 'WITH_PYMALLOC' is unset, Python ABI tag may be incorrect warn=(impl == 'cp')): Collecting pygame-1.9.2a0-cp35-none-win32 Could not find a version that satisfies the requirement pygame-1.9.2a0-cp35-none-win32 (from versions: ) No matching distribution found for pygame-1.9.2a0-cp35-none-win32 ** which tells me nothing. Please, someone - how do I install pygame? It appears to be installed on my PC - maybe I have it in the wrong folder? When I download the install packages, I copy them into my c:\Python folder (where my Python lives) and install from there so I would think they'd get installed in the right place. Many thanks - John Mycroft -- https://mail.python.org/mailman/listinfo/python-list
Re: Hello.
Environment variables pointing to c:\python..? (needs to point to actual installation directory) Jacob Kruger Blind Biker Skype: BlindZA "Roger Wilco wants to welcome you...to the space janitor's closet..." On 2016-01-16 11:41 PM, Hmood Js wrote: cmd won't recognize python at all I've checked several times , and I don't understand what's wrong Sent from Mail for Windows 10 -- https://mail.python.org/mailman/listinfo/python-list
Re: PyFladesk :: create GUI apps by Python and HTML, CSS and Javascript.
Ok, double-checked again, and if force it to run under 2.7, then it complains about lack of pyQT4 - that's one of the issues was asking about relating to GUI frameworks - pyQT hasn't always worked too well under 2.7 in terms of accessibility API's in past, but, will 'see' if can get hold of that module for 2.7, and take it from there. Jacob Kruger Blind Biker Skype: BlindZA "Roger Wilco wants to welcome you...to the space janitor's closet..." On 2016-01-08 7:07 AM, Bernardo Sulzbach wrote: On Fri, Jan 8, 2016 at 1:54 AM, jacob Kruger <ja...@blindza.co.za> wrote: Also, downloaded both the main master, and the RSS reader master images, but, both python 2.7 and python 3.4 tell me that they have no urllib2, and under 3.4, pip install can't find it either..? TIA Python 3 does not have urllib2. It is a Python 2 module that has been split across several modules in Python 3. However, are you sure you tried it with Python 2.7? -- https://mail.python.org/mailman/listinfo/python-list
Re: PyFladesk :: create GUI apps by Python and HTML, CSS and Javascript.
I would definitely like to try out something like this - I am primarily a web developer, and, partly since am 100% blind, any form of GUI design is at times an issue for me, whereas I have been working with HTML markup layouts for almost 20 years now, but, which versions of python should this work with, and, what does it actually then use to generate/handle GUI interface? Ask since, some of the GUI frameworks are not too accessible themselves. Also, downloaded both the main master, and the RSS reader master images, but, both python 2.7 and python 3.4 tell me that they have no urllib2, and under 3.4, pip install can't find it either..? TIA Jacob Kruger Blind Biker Skype: BlindZA "Roger Wilco wants to welcome you...to the space janitor's closet..." On 2016-01-08 1:08 AM, Saeed Moqadam wrote: create multi platform desktop application by using Python, HTML, CSS and Javascript. source code is https://github.com/smoqadam/PyFladesk you can find RSS Reader app that made by PyFladesk in the following url : https://github.com/smoqadam/PyFladesk-rss-reader I'll waiting for your feedback. thanks -- https://mail.python.org/mailman/listinfo/python-list
Re: Writing a python editor for blind developers
Germano, answering email at top of mail, since think that's the preferred method for some of us. I am also a 100% blind developer, and python is also one of my focus areas. While don't think can really help with this part of development as of yet, I would be more than willing to help with feedback, etc., and can also put you in touch with various other blind developers who do also work with python, etc. For example, AFAIK, the primary blind programmers mailing list is progra...@freelists.org, and besides me, there are a few other python developers on that list as well. Stay well Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... - Original Message - From: germano carella germanocarella.l...@gmail.com To: python-list@python.org Sent: Sunday, July 05, 2015 11:55 AM Subject: Writing a python editor for blind developers Hi to all, I'm new of this list. I'm Germano from Italy. I'm 39 and I'm a blind developer. I'm writing a python editor accessible to screen readers, with autocompletion support. So, when I write something, a context menu displays all option I can use. To do this, I'm using inspect module and pkgutil, and parsing docstring of builtin functions 'cause inspect.getargspec doesn't work with builtins. Now, when I instantiate a class, for example, I'd like to receive option on methods when I write name. ... I tired to use code.InteractiveConsole running in background. In this way, I can run source code every time I press enter and code.InteractiveConsole executes it in background. The problem is when I'm writing a function: InteractiveConsole executes it, but doesn't update her locals since I finish to write the function; so I can't retrieve local variables. There is an other way can you suggest me? Thanks! Germano -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Automation of Windows app?
- Original Message - From: Michael Torrie torr...@gmail.com To: python-list@python.org Sent: Monday, March 23, 2015 5:43 AM Subject: Re: Automation of Windows app? Nothing to do with Python, and its scripting language is maybe not that appealing to many, but it does the job, and does it pretty well. I first used it when working with a Linux-based PXE boot solution that would automatically install Windows via a network boot, and install a custom set of apps we set up. Was super slick. Also somewhat unrelated, but, my primary screen reader software, jaws, also allows you to generate scripts to sort of invoke multi-activity macro's with regards to operating system - so you could get it to click on a button, enter text in a field, move cursors around, etc. etc., repeatedly, if you wanted to - not really same as any form of bulk testing of software, etc., but, can interact with interfaces automatically, etc. Stay well Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... -- https://mail.python.org/mailman/listinfo/python-list
Re: Re: I want to get involved with Python!
- Original Message - From: bink...@mweb.co.za To: ja...@blindza.co.za Cc: python-list@python.org Sent: Wednesday, March 11, 2015 4:10 PM Subject: RE: Re: I want to get involved with Python! Jacob, thanks for replying. The IT job market is s specific in what they are looking for here in SA and it is really hard to get a job in a programming language one hasn't been exposed to. Even when you explain to the agency that you want to learn new technologies and that the only way you can acquire it is if someone gives you a chance. The recruitment agencies blatantly block your CV for those positions and it never gets to the clients. Bianca, I know that too well - however, the one reason I got my current position was that while it primarily involves having PHP/web development abilities, they also specifically wanted a broader range of areas of experience, specifically including python, along with a couple of other languages/platforms, etc., but, it's not too common in that sense. Alternatively, UCT has recently started offering forms of online, location non-specific courses, relating to forms of certification qualifications, as well as learning, and think they might include offering some courses relating to python, etc. - and, while it's a form of remote learning, it is also a local education provider..? But, yes, am sure some of the other list members will have more to offer - I myself have no real formal training, and work in a slightly odd sector in terms of my own approach - but, will send you a bit more information specific to local python activity, off list. Stay well Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... -- https://mail.python.org/mailman/listinfo/python-list
Re: I want to get involved with Python!
- Original Message - From: bink...@mweb.co.za To: python-list@python.org Sent: Wednesday, March 11, 2015 11:19 AM Subject: I want to get involved with Python! Please advise on what route you would suggest I take to become a Python developer! Thanks, Bianca. Bianca, best bet to check out availability of positions in SA is to check out online job listing websites, like careers24, pnet and jobmail, etc. And, while have no idea about other forms of training, I just got into python by working through various forms of online tutorial material, etc., as well as working with some of the eBooks available - but, sure some of the other list members will have better/more specific suggestions in that regard. And, yes, I work in/with python, but, my other primary field of focus here in SA is web development, so not too sure about industry specific use of different languages/platforms. Stay well Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... -- https://mail.python.org/mailman/listinfo/python-list
Re: Accessible tools
- Original Message - From: Tim Chase python.l...@tim.thechases.com Subject: Re: Accessible tools While my experience has shown most of your items to be true, I'd contend that • Do not, have access to debugging tools. is mistaken or at least misinformed. For Python, I use the pdb module all the time, and it's command-line driven. Combined with a multi-terminal (whether multiple windows, virtual consoles, or a tmux/screen session), I can easily bounce back and forth between a pdb debugging session and the source code to make edits. Just to check, I fired up the yasr terminal screen-reader, launched tmux (using my quiet config, since it updates information on the screen like the time on a regular basis, making it chatty), and stepped through some Python code, checked variables, and walked up/down the call-stack. I know most other languages have similar functionality such as gdb for C code. Will check out PDB a bit more, but, honestly, my windows screenreader that use most of the time, jaws, doesn't always cooperate perfectly with command line/console interface - can be worked around, but, not all that easily at times - but, this page seems to offer enough detail relating to PDB, to start off with anyway: https://docs.python.org/3/library/pdb.html Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... -- https://mail.python.org/mailman/listinfo/python-list
Re: Accessible tools
Eric, issue is that with screenreaders, we're generally way more into navigating code and interface character by character/by keyboard, so , yes, keeping interface relatively simple is a good thing, but, we also would prefer to primarily keep all interface elements to make use of standard UI controls, and make sure tab index/order is suitable/relevant at times, etc. etc. As in, I think we'd primarily want to avoid having to use a mouse at all if possible, but anyway. Stay well Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... - Original Message - From: Eric S. Johansson e...@harvee.org To: python-list@python.org Sent: Friday, February 20, 2015 7:22 PM Subject: Re: Accessible tools On 2/19/2015 10:33 AM, Bryan Duarte wrote: Thank you jwi, and Jacob, I took a look at that posting and it seems pretty unique. I am not much interested in the speech driven development, but I am very interested in developing an accessible IDE. Well you should be because it looks like an aural interface (uses speech instead of keyboards) uses the same kinds of data to present to either a text to speech or speech recognition driven environment. A professor and I have been throwing around the idea of developing a completely text based IDE. There are a lot of reasons this could be beneficial to a blind developer and maybe even some sighted developers who are comfortable in the terminal. The idea would be really just to provide a way of easily navigating blocks of code using some kind of tabular formatting, and being able to collapse blocks of code and hearing from a high level information about the code within. All tools and features would obviously be spoken or output in some kind of audio manor. I've been working with another professor working on some of these issues as well. His focus has been mostly blind young adults in India. come up with some pretty cool concepts that looks very usable. The challenge now is to make them work and, quite frankly monetize the effort to pay for the development. Again, this shows the similarities in functionality used by both speech recognition and text-to-speech. All I care about is text and what I can say. We're now working with constructs such as with-open, argument by number, plaintext symbol names (with bidirectional transform to and from code form), guided construct generation for things like classes, methods, comprehensions etc. All of these things would be useful to handed programmers as well as a way of accelerating co-creation and editing. Unfortunately, like with disabled people stove piping text-to-speech versus speech recognition, handed developers stovepipe keyboard interfaces and don't really think about what they are trying to do, only how they are doing it. Yes yes, it's a broadbrush that you can probably slap me with. :-) Oh and before I forget does anyone know how to contact Eric who was developing that accessible speech driven IDE? Thanks Well, you could try looking in a mirror and speaking my name three times at midnight But you would get better results if you used my non-mailing list email address. e...@eggo.org. --- eric -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Accessible tools
Brian, I am also a fully bind python/PHP developer, and all I really bother with, aside from python interpreter itself is the text editor called edSharp: http://www.empowermentzone.com/edsetup.exe That's been developed by another blind guy who's name you might have heard in other circles, Jamal Mazrui, and it's just a programmers text editor that has a bit of extra functionality built in for use along with screen readers like jaws and NVDA, etc. And, there are also some other general blind programmer mailing lists - I am on both progra...@freelists.org as well as a couple of other topic specific ones, but anyway. Stay well Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... - Original Message - From: Bryan Duarte bryan0...@gmail.com To: python-list@python.org Sent: Wednesday, February 18, 2015 7:43 PM Subject: Accessible tools Hello all, I have been posting to another group which directed me to this group. I am a blind software engineering student at Arizona State University. I am currently doing research and have decided to use Python as my developing language. I am in search of an accessible IDE or other tool set which will allow me to use the features with a screen reader. I have tried a number of tools but so far have been unsuccessful in finding one that allows me to use the tools and features of the IDE with my screen reader. I understand that I do not need an IDE to use Python, I am looking for an IDE to allow me to explore the classes and methods easily and quickly. I have tried the following tools. • pyCharm • iPy • iPy Notebook • Idle • and Eclipse So far only Eclipse was accessible but I did still incounter some difficulties like the IDE freezing up when tool tips would pop up. Finally I am using a Mac mostly but can use Windows, or Linux if need be. Thank you and I appreciate any and all suggestions. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Accessible tools
I use edSharp as my primary code/text editor, and just save code from there, and while it doesn't offer too many things like design tools, it's pretty much perfect in terms of code block selection, code snippets, code navigation - to a certain extent, even in python, with regards to jumping from block to block, etc. Text based IDE would be nice at times, as long as you could review code nicely enough, etc., and in terms of GUI design I generally just make use of/work with scrollable grids via wxPython, since that cooperates well enough in end result with both sighted, and accessibility API users, etc., but anyway. It just means my interface design is done via a form of layout-by-code, dropping controls in a flexGridSizer as well, but anyway. My one primary thing I would like to be able to handle in terms of development process is better means of working with debugging code execution, etc. - at moment, primarily either print out information to console, during dev process, or at times invoke sound effects/TTS output to keep track, or even, occasionally use pickle to store images of objects for later review, etc., but anyway...smile Stay well Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... - Original Message - From: Bryan Duarte bryan0...@gmail.com To: Jonas Wielicki jo...@wielicki.name Cc: python-list@python.org Sent: Thursday, February 19, 2015 5:33 PM Subject: Re: Accessible tools Thank you jwi, and Jacob, I took a look at that posting and it seems pretty unique. I am not much interested in the speech driven development, but I am very interested in developing an accessible IDE. A professor and I have been throwing around the idea of developing a completely text based IDE. There are a lot of reasons this could be beneficial to a blind developer and maybe even some sighted developers who are comfortable in the terminal. The idea would be really just to provide a way of easily navigating blocks of code using some kind of tabular formatting, and being able to collapse blocks of code and hearing from a high level information about the code within. All tools and features would obviously be spoken or output in some kind of audio manor. Jacob, I know your name and I do know Jamal's name as well. I think I recall your names from either back in the Mobile Speak days, or maybe from the jaws mailing list. Either way thank you for the feedback and I will take a look at edSharp today. The Python interpreter is great for small tests or scripts but for lengthy programs there is no easy way to save your code other than capturing the entire history with extra code and all. How do you typically handle that issue? Thank you both. Oh and before I forget does anyone know how to contact Eric who was developing that accessible speech driven IDE? Thanks On Feb 19, 2015, at 3:08 AM, Jonas Wielicki jo...@wielicki.name wrote: Dear Bryan, I don’t have a finished solution for you, but I would like to link you to a previous thread on this list: https://mail.python.org/pipermail/python-list/2015-January/696276.html The poster seems to be researching into the direction of developing a speech-friendly IDE. You may want to follow his work. regards, jwi -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
help() function screen output
Using python 3.4 32 bit on windows 7 64 bit machine, and when, for example, type in something like the following in interpreter window: help(str) It will populate the screen with one full screen of information, with a prompt of --more-- to hit enter, or something to continue displaying information, but, when, for example, hit enter key, it merely populates one more line of info, each time - under pythn 2.7, it seemed to render one full screen of info each time. Is there rather a different keystroke should use to invoke that, or is this a python 3.4 feature/issue? TIA Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... -- https://mail.python.org/mailman/listinfo/python-list
Re: help() function screen output
- Original Message - Specifically, the last two lines show: space Display next page ret Display next line which is, I suspect, what you're after here. (If you're interested, the code for all this is in the pydoc module (Lib\pydoc.py) in the getpager() function. TJG -- https://mail.python.org/mailman/listinfo/python-list Yes, that makes sense/works - wonder what changed recently in my overall configuration/setup - suppose might have something to do with screen reader's system wide interference - maybe it was passing a different effective keystroke through to command line prompt before, or something, since am 99.9% sure always just used ret key to advance one screen at a time before - who knows, but, main thing is, yes, space bar does what I want it to. Thanks Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... -- https://mail.python.org/mailman/listinfo/python-list
Re: help() function screen output
- Original Message - From: Steven D'Aprano steve+comp.lang.pyt...@pearwood.info Newsgroups: comp.lang.python To: python-list@python.org Sent: Saturday, January 31, 2015 11:13 AM Subject: Re: help() function screen output Jacob Kruger wrote: Using python 3.4 32 bit on windows 7 64 bit machine, and when, for example, type in something like the following in interpreter window: help(str) It will populate the screen with one full screen of information, with a prompt of --more-- to hit enter, or something to continue displaying information, but, when, for example, hit enter key, it merely populates one more line of info, each time - under pythn 2.7, it seemed to render one full screen of info each time. Are you sure? I don't have Windows to try, but in Linux using both Python 2.7 and 3.3, pressing Enter advances by a single line. Is there rather a different keystroke should use to invoke that, or is this a python 3.4 feature/issue? Try using the Space key to advance an entire page. You can also try Page Up and Page Down keys. Ok, strangely, while am relatively sure 2.7 rendered a screen at a time before when hitting enter key, it now also seems to advance one line at a time, but, yes, seems like space bar does advance a whole screen in both versions, and page up/down doesn't react, but, space bar will do for now...smile And, I generally let it render a screen full, and then review the screen contents using screen-reader screen review functionality, but, in this case it pretty much matches what people see. Thanks Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... -- https://mail.python.org/mailman/listinfo/python-list
Re: [OT] Re: MySQL connections
- Original Message - From: Dennis Lee Bieber wlfr...@ix.netcom.com To: python-list@python.org Sent: Thursday, January 15, 2015 4:22 PM Subject: Re: [OT] Re: MySQL connections On Thu, 15 Jan 2015 13:48:34 +0200, Jacob Kruger ja...@blindza.co.za declaimed the following: Agree with that, but, like said in prior e-mail, just get windows error message popping up, mentioning - no track trace, since it's python terminating: Python is fairly good about dumping a stack trace when the error is something detectable by Python. Issue is it just kills python, so no error message from python being rendered/displayed. Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... -- https://mail.python.org/mailman/listinfo/python-list
Re: MySQL connections
- Original Message -
From: Chris Angelico ros...@gmail.com
Cc: python-list@python.org
Sent: Thursday, January 15, 2015 2:33 PM
Subject: Re: MySQL connections
On Thu, Jan 15, 2015 at 10:59 PM, Jacob Kruger ja...@blindza.co.za
wrote:
Tried generating .exe with both cx_freeze, and pyInstaller, and the code
itself, and both versions of executable generate errors, same as running
it
from command line - only difference is the source of the error mentioned
in
error message then varies from a2m.exe and python.exe , and even if
compile
it on machine where all works, and then copy .exe back to this primary
machine, then get same error - think it must relate to something else on
this machine, but can't track it down.
Okay. Ignore the .exe versions, and just work with what happens when
you run the .py files. If it fails as part of a .py file, post the
failing file.
If you want to check it out, have attached the full code file - might be a
bit messy/large - but, effectively, right at bottom, launch an instance of
the class a2m, passing through arguments, and then from within __init__ call
convertProcess function, which then initiates process, harvesting sort of
rendition/version of structure out of MS access database file, makes call to
convertSQL to generate structural SQL script, and save it to a file, which
then calls convertData function to generate insert statements to populate
database, and then that makes a call to convertExport, if you passed a
command line argument in requesting mysql, and that's the current issue
function - have stripped most of actual functionality out of it, since am
just testing, so the first 16 lines of that function are what's relevant at
moment - think shouldn't rely on any other self. objects/attributes as such
either, unless step-through process is an issue.
And, tried cleaning it up a bit - replaced tab indentation character with
double spaces, but, code might not look perfect/be perfectly clean as of
yet.
Jacob Kruger
Blind Biker
Skype: BlindZA
Roger Wilco wants to welcome you...to the space janitor's closet...
import pypyodbc, os, sys
import copy, warnings
import pymysql, time, pickle
from unidecode import unidecode
def sTimeDiffFormat(i_time1, i_time2):
i_total_time = i_time2 - i_time1 if i_time2 i_time1 else i_time1 - i_time2
i_total_time = int(i_total_time)
#time.strftime(%H:%M:%S, time.gmtime(i_total_time))
i_hours = 0
i_minutes = 0
if i_total_time = 3600:
i_hours = int(i_total_time / 3600)
i_total_time = i_total_time - (i_hours * 3600)
if i_total_time 60:
i_minutes = int(i_total_time/60)
i_total_time = i_total_time - (i_minutes * 60)
s_out =
if i_hours 0: s_out = {0} hours .format(i_hours)
if i_minutes 0: s_out = s_out + {0} minutes .format(i_minutes)
s_out = s_out + {0} seconds.format(i_total_time)
return s_out
#end of sTimeDiffFormat
class a2m():
s_mdb =
s_mdb_pass =
s_mdb_conn =
bl_mdb_conn = False
d_db_tables = {}
cn_mdb = None
s_structure_sql =
d_db_data = {}
l_field_types = []
s_target =
s_target_value =
s_sql_struct_file =
s_sql_data_file =
def __init__(self, s_mdb, s_mdb_pass, s_target, s_target_value):
self.s_mdb = str(s_mdb) if len(str(s_mdb)) 0 else
self.s_mdb_pass = str(s_mdb_pass) if len(str(s_mdb_pass)) 0 else
if self.s_mdb_pass == #: self.s_mdb_pass =
self.s_target = str(s_target) if len(str(s_target))0 else
self.s_target_value = str(s_target_value) if len(str(s_target_value))0 else
self.s_mdb_conn =
self.bl_mdb_conn = False
if self.s_mdb != and self.s_target != and self.s_target_value != :
self.convertProcess()
#end of __init__
def shiftID(self, l_columns):
if l_columns.count(ID)0:
l_columns.remove(ID)
l_columns.insert(0, ID)
return l_columns
#end of shiftID
def dictDBStructure(self, cur):
d_out = {}
cur_tables = cur.tables(tableType=table).fetchall()
s_tables =
for l_table in cur_tables:
s_tables += str(l_table[2]) + |
l_tables = s_tables.split(|)
if len(l_tables) 1: l_tables.pop(len(l_tables)-1)
for s_table in l_tables:
d_out[s_table] = {}
#s_columns =
#INTEGER, SMALLINT, VARCHAR, LONGCHAR, CURRENCY, BIT, DATETIME, COUNTER?
for col in cur.columns(table=s_table):
s_col_name = str(col.get(column_name))
s_col_type = str(col.get(type_name))
i_col_size = int(col.get(column_size))
if list(d_out[s_table].keys()).count(s_col_name) 1:
d_out[s_table][s_col_name] = {column_name: s_col_name, column_type:
s_col_type, column_size: i_col_size}
return d_out
#end of dictDBStructure
def pullDataTypes(self, evt):
if len(self.s_mdb) 0:
s_path = self.s_mdb
s_pwd = self.DialogInput(title=Password, label=Password (if required):,
value=)
if len(s_pwd) 0: s_pwd = pwd={0};.format(s_pwd)
self.s_mdb_conn = Driver=Microsoft Access Driver (*.mdb, *.accdb); + s_pwd +
DBQ= + s_path
try:
self.cn_mdb
Re: [OT] Re: MySQL connections
- Original Message - functions. When was the last time those systems had Windows Update and reboots performed? Daily basis. Think, in line with your other message, will just try rewrite code - and, issue relating to structure etc. is left over from when pulled it out of wxPython implementation - you're right that should have just redone that part of it ;) Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... -- https://mail.python.org/mailman/listinfo/python-list
MySQL connections
Development machine is windows7 64 bit machine, now working with either/both python 2.7 or 3.4 - test purposes, trying to figure out if this was something like a version incompatibility issue, and tried using both XAMPP and WAMP MySQL server version 5.0.11 instances thus far. Now, aside from all other issues relating to character encoding, etc., now down to, hopefully, final issue. Side note is under python 2.7, I use pyodbc to gather data from .mdb file am using to test, but under python 3.4, I use pypyodbc for same functionality. Now, current/actual issue is something can sort of duplicate using both MySQLdb version of mysqlclient, as well as pymysql using both python 2.7 and python 3.4, but, strange thing am trying to figure out is that even if just implement a connection object in interpreter of either version, retrieve an instance of a cursor object, and do various types of statement executions, no issues, and can then close connection, or cursor, or both, and all good/fine/well. However, if implement similar code - down to just trying to open a connection, wait a few seconds, and then close it again, inside a function called from a prior function, in the class am implementing in a file called/executed from command line, then, the moment I try to close the connection, whether or not have done anything with it, that's when I get that python.exe has stopped working/responding error message popping up, and python bombs out. Have no idea how to try track down actual source/cause of issue, since haven't found anything obvious in system event logs, nothing appearing in MySQL server logs that makes sense/appears relevant, and no matter if I try enclosing the connection open/close code inside try except code blocks, nothing happens there except that python just stops cooperating at all, and stops working. Now did try making sure data execution prevention was turned on/off, and that if set to second mode then told it to ignore specific applications - added both python.exe, and the compiled version of my code to it's exemption list, but, no changes there. This still operates fine on my other dev machine, but, would like to know why it's not operating on one specific machine. Otherwise, will just have to tell guys that, sorry, will only be able to generate SQL script text files, and they'll have to stick to the final step of then executing those via MySQL console interface, or something, but, would really prefer to at the very least, figure/find out what's causing this mis-behaviour. Thoughts/suggestions with regards to how to find out what is happening/what I'm doing/handling wrong/incorrectly? The two current versions of the code - which work fine under interpreter of both versions of python are basically the following: #pymysql version import pymysql cn = pymysql.connect(localhost, root, , import_test) time.sleep(5) cn.close() #MySQLdb version import MySQLdb cn = MySQLdb.connect(localhost, root, , import_test) time.sleep(5) cn.close() #end code Stay well (away from this machine ;) ) Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... -- https://mail.python.org/mailman/listinfo/python-list
Re: MySQL connections
And, FWIW, if I compile the 2.7 version on the other machine where it works, in both code and compiled forms, and then copy .exe back to the main machine, same error message pops up, so must be something to do with machine's configuration, etc. Stay well Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... - Original Message - From: Jacob Kruger To: python-list@python.org Sent: Thursday, January 15, 2015 10:13 AM Subject: MySQL connections Development machine is windows7 64 bit machine, now working with either/both python 2.7 or 3.4 - test purposes, trying to figure out if this was something like a version incompatibility issue, and tried using both XAMPP and WAMP MySQL server version 5.0.11 instances thus far. Now, aside from all other issues relating to character encoding, etc., now down to, hopefully, final issue. Side note is under python 2.7, I use pyodbc to gather data from .mdb file am using to test, but under python 3.4, I use pypyodbc for same functionality. Now, current/actual issue is something can sort of duplicate using both MySQLdb version of mysqlclient, as well as pymysql using both python 2.7 and python 3.4, but, strange thing am trying to figure out is that even if just implement a connection object in interpreter of either version, retrieve an instance of a cursor object, and do various types of statement executions, no issues, and can then close connection, or cursor, or both, and all good/fine/well. However, if implement similar code - down to just trying to open a connection, wait a few seconds, and then close it again, inside a function called from a prior function, in the class am implementing in a file called/executed from command line, then, the moment I try to close the connection, whether or not have done anything with it, that's when I get that python.exe has stopped working/responding error message popping up, and python bombs out. Have no idea how to try track down actual source/cause of issue, since haven't found anything obvious in system event logs, nothing appearing in MySQL server logs that makes sense/appears relevant, and no matter if I try enclosing the connection open/close code inside try except code blocks, nothing happens there except that python just stops cooperating at all, and stops working. Now did try making sure data execution prevention was turned on/off, and that if set to second mode then told it to ignore specific applications - added both python.exe, and the compiled version of my code to it's exemption list, but, no changes there. This still operates fine on my other dev machine, but, would like to know why it's not operating on one specific machine. Otherwise, will just have to tell guys that, sorry, will only be able to generate SQL script text files, and they'll have to stick to the final step of then executing those via MySQL console interface, or something, but, would really prefer to at the very least, figure/find out what's causing this mis-behaviour. Thoughts/suggestions with regards to how to find out what is happening/what I'm doing/handling wrong/incorrectly? The two current versions of the code - which work fine under interpreter of both versions of python are basically the following: #pymysql version import pymysql cn = pymysql.connect(localhost, root, , import_test) time.sleep(5) cn.close() #MySQLdb version import MySQLdb cn = MySQLdb.connect(localhost, root, , import_test) time.sleep(5) cn.close() #end code Stay well (away from this machine ;) ) Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... -- -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: [OT] Re: MySQL connections
- Original Message - From: Peter Otten __pete...@web.de To: python-list@python.org Sent: Thursday, January 15, 2015 12:40 PM Subject: [OT] Re: MySQL connections Jacob Kruger wrote: However, if implement similar code - down to just trying to open a connection, wait a few seconds, and then close it again, inside a function called from a prior function, in the class am implementing in a file called/executed from command line, then, the moment I try to close the connection, whether or not have done anything with it, that's when I get that python.exe has stopped working/responding error message popping up, and python bombs out. Jacob, your writing style is incredibly long-winded and hard to follow. Try to build shorter sentences instead of this stream of consciousness, and focus on the actual problem. Was just trying to make sure included all relevant details ;) python bombs out is not a meaningful problem description. A traceback might be. Agree with that, but, like said in prior e-mail, just get windows error message popping up, mentioning - no track trace, since it's python terminating: ---error details--- Problem Event Name: BEX Application Name: python.exe Fault Module Name: StackHash_0a9e ---end error details--- That term BEX equates to buffer overflow exception, but not sure why this is happening - previous suggestion was to do with data execution prevention on windows, but, bex apparently also relates to winsock, or something, and tried different types of applying DEP - think so anyway. That said, I probably cannot help with your particular problem, so feel free to ignore me... No worries - thanks for reply. Just found following bit of summary of possible combination of bex/stackhash - but, still doesn't make sense when at this test stage I'm literally just opening and closing connection - might make sense if was executing thousands of queries against database before committing them: ---quote--- Buffer overflow is a condition when some process tries to store data beyond the capacity of the fixed/available buffer so it tries to overwrite some other memory locations, too. And in Windows we have some security feature called Data Execution Prevention that is intended to prevent similar processes to prevent buffer overflow attacks (that can introduce some malicious codes). But in some cases DEP can prevent some legitimate software from executing, too. And then you can get a BEX error. ---end quote--- Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... -- https://mail.python.org/mailman/listinfo/python-list
Re: MySQL connections
- Original Message - From: Chris Angelico ros...@gmail.com You've posted your working versions; can you post a non-working version? Problem is that works fine in interpreter, but, not when executing it as part of code in file. Also, I'm seeing a very small hint here that might indicate a huge factor that you haven't mentioned. You speak of compiling your Python code. Do you mean you're turning it into an exe file? If so, with which tool? Does the code still have problems if you run the actual Python file? Tried generating .exe with both cx_freeze, and pyInstaller, and the code itself, and both versions of executable generate errors, same as running it from command line - only difference is the source of the error mentioned in error message then varies from a2m.exe and python.exe , and even if compile it on machine where all works, and then copy .exe back to this primary machine, then get same error - think it must relate to something else on this machine, but can't track it down. Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 2.7, on windows7 64 bit development machine, inconsistent issue on similar machines
Ok, and just tested MySQLdb connection to both XAMPP server instance on same machine, as well as slightly remote connection to other machine over wifi, and same error - so, seems issue is invoked/caused by MySQLdb connection closing - if just put process to sleep for 30 seconds, nothing happens, but, the moment I in fact try to then just close the connection, that's when error dialogue immediately pops up. Sorry - should also have mentioned the connection, cursor, etc. ec. all work fine under normal python interpreter, but anyway. Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 2.7, on windows7 64 bit development machine, inconsistent issue on similar machines
See answers below.
Jacob Kruger
Blind Biker
Skype: BlindZA
Roger Wilco wants to welcome you...to the space janitor's closet...
- Original Message -
From: Dennis Lee Bieber wlfr...@ix.netcom.com
snip /
The very first hit /I/ get is:
https://social.technet.microsoft.com/Forums/windows/en-US/3932e3eb-c034-4eb7-aa06-4a0a8e6ea493/fault-module-namestackhash0a9e
Played around with data expraction policy changes/settings now, and didn't
make it change behaviour.
snip /
s_host = str(d_mysql[host])
s_user = str(d_mysql[user])
s_pass = str(d_mysql[password])
s_db = self.s_target_value
ONE: Have you confirmed all of these are coming in correctly
Yes - that mysql.set pickle dump is just a way of storing some string values
relating to mysql server connection to sort of hide it from an end-user,
but, yes, the values work fine in that sense.
cn = MySQLdb.connect(s_host, s_user, s_pass, s_db)
if cn:
cn.autocommit = True
TWO-A: you've turned on autocommit here, yet...
Sorry - that was sort of left-over from when was trying to make sure that
other parts were happening/working - along with trying out specific commit
call lower down - have tried taking eitehr of them out one by one, to see if
made a difference, but, no-go.
cur = cn.cursor()
i_struct = 0
i_data = 0
f = open(self.s_sql_struct_file, rb)
s = f.read()
f.close()
l_struct = s.split(;)
THREE-A: here you split some string on semicolons, just to...
f = open(self.s_sql_data_file, rb)
s = f.read()
f.close()
l_data = unidecode(s).split(;)
l_struct_errors = []
l_data_errors = []
i_start_time = int(time.time())
s_status = /{0} structs.format(str(len(l_struct)))
for I in range(len(l_struct)):
Very UN-Pythonic...
I know - that was also since originally used this code in a sort of GUI
version, and meant to use one by one counter to update status bar text at
sort of intervals of 10 records, to let guys track progress - but, FWIW, it
seemed to overload my screen reader with too many changes, partly since some
of these collections of insert statements are processing over 4 record
insertions - not currently with test data, but, some of the other test
database files I used were operating in that region of the number of data
record numbers. Have taken that part out again now.
for a_struct in l_struct:
# do stuff with a_struct directly, no indexing into l_struct
try:
i_struct = I
if str(l_struct[I]).strip() != : res =
cur.execute(l_struct[I] + ;)
THREE-B: ... reappend the semicolon here... which probably isn't needed
-- cur.execute() assumes, as I recall, that the provided argument is a
complete statement; unlike a command line interface where the semicolon is
needed to tell the CLI that the statement is complete and can be sent to
the DBMS for processing.
And very confusing names... l_struct and i_struct, where the latter is
an index into the former (and I don't see it used anywhere).
Have taken out counter now, and, yes, cursor seems happy to execute
statements without ; character at end.
print(l_struct[I])
#except Warning as wrn:
# print(warning: + str(wrn.args))
except Exception as exc:
l_struct_errors.append([l_struct[I], copy.copy(exc)])
finally:
pass
cn.commit()
TWO-B: ... here you attempt to force a commit.
That was also during test phase when was just trying to make sure parts of
this were being executed/committed - taken out additional commits now.
time.sleep(2.0)
sNada = raw_input(hit enter to continue with data)
s_status = /{0} data.format(str(len(l_data)))
for I in range(len(l_data)):
try:
#self.SetStatusText(str(I) + s_status)
i_data = I
if str(l_data[I]).strip() != : res =
cur.execute(l_data[I][l_data[I].index(INSERT):] + ;)
print(l_data[I][l_data[I].index(INSERT):] + ;)
except Exception as exc:
l_data_errors.append([l_data[I], copy.copy(exc)])
finally:
pass
i_end_time = int(time.time())
s_time_taken = sTimeDiffFormat(i_start_time, i_end_time)
cn.commit()
TWO-C: ... and here too.
cur.close()
cn.close()
print(cn and cur closed)
fPickle = open(testDataErrors.pickle, wb)
pickle.dump(l_struct_errors, fPickle, 2)
pickle.dump(l_data_errors, fPickle, 2)
fPickle.close()
print(pickled)
print(MySQL Results - {0} structure queries, and {1} data queries
completed - total of {2}.format(str(i_struct), str(i_data),
s_time_taken))
print(MySQL Errors - {0} structure query errors, and {1} data
query errors.format(str(len(l_struct_errors)), str(len(l_data_errors
print(Done!)
sys.exit()
else:
print
Re: Python 2.7, on windows7 64 bit development machine, inconsistent issue on similar machines
Ok, and just tested MySQLdb connection to both XAMPP server instance on same
machine, as well as slightly remote connection to other machine over wifi,
and same error - so, seems issue is invoked/caused by MySQLdb connection
closing - if just put process to sleep for 30 seconds, nothing happens, but,
the moment I in fact try to then just close the connection, that's when
error dialogue immediately pops up.
Stay well
Jacob Kruger
Blind Biker
Skype: BlindZA
Roger Wilco wants to welcome you...to the space janitor's closet...
- Original Message -
From: Jacob Kruger ja...@blindza.co.za
To: Dennis Lee Bieber wlfr...@ix.netcom.com
Cc: python-list@python.org
Sent: Tuesday, January 13, 2015 8:37 PM
Subject: Re: Python 2.7, on windows7 64 bit development machine,
inconsistent issue on similar machines
See answers below.
Jacob Kruger
Blind Biker
Skype: BlindZA
Roger Wilco wants to welcome you...to the space janitor's closet...
- Original Message -
From: Dennis Lee Bieber wlfr...@ix.netcom.com
snip /
The very first hit /I/ get is:
https://social.technet.microsoft.com/Forums/windows/en-US/3932e3eb-c034-4eb7-aa06-4a0a8e6ea493/fault-module-namestackhash0a9e
Played around with data expraction policy changes/settings now, and didn't
make it change behaviour.
snip /
s_host = str(d_mysql[host])
s_user = str(d_mysql[user])
s_pass = str(d_mysql[password])
s_db = self.s_target_value
ONE: Have you confirmed all of these are coming in correctly
Yes - that mysql.set pickle dump is just a way of storing some string
values relating to mysql server connection to sort of hide it from an
end-user, but, yes, the values work fine in that sense.
cn = MySQLdb.connect(s_host, s_user, s_pass, s_db)
if cn:
cn.autocommit = True
TWO-A: you've turned on autocommit here, yet...
Sorry - that was sort of left-over from when was trying to make sure that
other parts were happening/working - along with trying out specific commit
call lower down - have tried taking eitehr of them out one by one, to see
if made a difference, but, no-go.
cur = cn.cursor()
i_struct = 0
i_data = 0
f = open(self.s_sql_struct_file, rb)
s = f.read()
f.close()
l_struct = s.split(;)
THREE-A: here you split some string on semicolons, just to...
f = open(self.s_sql_data_file, rb)
s = f.read()
f.close()
l_data = unidecode(s).split(;)
l_struct_errors = []
l_data_errors = []
i_start_time = int(time.time())
s_status = /{0} structs.format(str(len(l_struct)))
for I in range(len(l_struct)):
Very UN-Pythonic...
I know - that was also since originally used this code in a sort of GUI
version, and meant to use one by one counter to update status bar text at
sort of intervals of 10 records, to let guys track progress - but, FWIW,
it seemed to overload my screen reader with too many changes, partly since
some of these collections of insert statements are processing over 4
record insertions - not currently with test data, but, some of the other
test database files I used were operating in that region of the number of
data record numbers. Have taken that part out again now.
for a_struct in l_struct:
# do stuff with a_struct directly, no indexing into l_struct
try:
i_struct = I
if str(l_struct[I]).strip() != : res =
cur.execute(l_struct[I] + ;)
THREE-B: ... reappend the semicolon here... which probably isn't needed
-- cur.execute() assumes, as I recall, that the provided argument is a
complete statement; unlike a command line interface where the semicolon
is
needed to tell the CLI that the statement is complete and can be sent to
the DBMS for processing.
And very confusing names... l_struct and i_struct, where the latter is
an index into the former (and I don't see it used anywhere).
Have taken out counter now, and, yes, cursor seems happy to execute
statements without ; character at end.
print(l_struct[I])
#except Warning as wrn:
# print(warning: + str(wrn.args))
except Exception as exc:
l_struct_errors.append([l_struct[I], copy.copy(exc)])
finally:
pass
cn.commit()
TWO-B: ... here you attempt to force a commit.
That was also during test phase when was just trying to make sure parts of
this were being executed/committed - taken out additional commits now.
time.sleep(2.0)
sNada = raw_input(hit enter to continue with data)
s_status = /{0} data.format(str(len(l_data)))
for I in range(len(l_data)):
try:
#self.SetStatusText(str(I) + s_status)
i_data = I
if str(l_data[I]).strip() != : res =
cur.execute(l_data[I][l_data[I].index(INSERT):] + ;)
print(l_data[I][l_data[I].index(INSERT):] + ;)
except Exception as exc
Python 2.7, on windows7 64 bit development machine, inconsistent issue on similar machines
#---end code--- If this is operating system specific, sorry, and will maybe also try debugging code via VS.Net 2013 shortly, but, thought someone might have an off-hand thought regarding something could try use to figure out this issue in meantime...? TIA Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... -- https://mail.python.org/mailman/listinfo/python-list
unidecode
Along lines of recent queries/questions, which had mostly sorted out, but, which was still trying to fine tune with regards to text file formats, in terms of writing as strings, or as binary files, and where was still trying to decide if wanted to worry about maybe specifically just handling forms of manual/hard-coded translations of certain accented characters which might relate to some of them that might get used quite often in names, etc., I just came across the following unidecode module: https://pypi.python.org/pypi/Unidecode/ Seems to just take any unicode value, and 'translate' it into a string value, and will apparently try handle some forms of character substitution - but will play around with it a bit more, and let's see... Stay well Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... - Original Message - From: Jacob Kruger To: Jacob Kruger Sent: Monday, January 12, 2015 2:15 PM Subject: unidecode https://pypi.python.org/pypi/Unidecode/ Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... -- https://mail.python.org/mailman/listinfo/python-list
String character encoding when converting data from one type/format to another
I'm busy using something like pyodbc to pull data out of MS access .mdb files,
and then generate .sql script files to execute against MySQL databases using
MySQLdb module, but, issue is forms of characters in string values that don't
fit inside the 0-127 range - current one seems to be something like \xa3, and
if I pass it through ord() function, it comes out as character number 163.
Now issue is, yes, could just run through the hundreds of thousands of
characters in these resulting strings, and strip out any that are not within
the basic 0-127 range, but, that could result in corrupting data - think so
anyway.
Anyway, issue is, for example, if I try something like
str('\xa3').encode('utf-8') or str('\xa3').encode('ascii'), or
str('\xa3').encode('latin7') - that last one is actually our preferred encoding
for the MySQL database - they all just tell me they can't work with a character
out of range.
Any thoughts on a sort of generic method/means to handle any/all characters
that might be out of range when having pulled them out of something like these
MS access databases?
Another side note is for binary values that might store binary values, I use
something like the following to generate hex-based strings that work alright
when then inserting said same binary values into longblob fields, but, don't
think this would really help for what are really just most likely badly chosen
copy/pasted strings from documents, with strange encoding, or something:
#sample code line for binary encoding into string output
s_values += 0x + str(l_data[J][I]).encode(hex).replace(\\, ) + ,
TIA
Jacob Kruger
Blind Biker
Skype: BlindZA
Roger Wilco wants to welcome you...to the space janitor's closet...
--
https://mail.python.org/mailman/listinfo/python-list
Re: String character encoding when converting data from one type/format to another
Thanks.
Makes more sense now, and yes, using 2.7 here.
Unfortunately, while could pass the binary values into blob fields well
enough, using forms of parameterised statements, the actual generation of
sql script text files is a step they want to work with at times, if someone
is handling this on site, so had to work first with generating string
values, and then handle executing those statements against a MySQL server
later on using MySQLdb.
Stay well
Jacob Kruger
Blind Biker
Skype: BlindZA
Roger Wilco wants to welcome you...to the space janitor's closet...
- Original Message -
From: Peter Otten __pete...@web.de
To: python-list@python.org
Sent: Wednesday, January 07, 2015 2:11 PM
Subject: Re: String character encoding when converting data from one
type/format to another
Jacob Kruger wrote:
I'm busy using something like pyodbc to pull data out of MS access .mdb
files, and then generate .sql script files to execute against MySQL
databases using MySQLdb module, but, issue is forms of characters in
string values that don't fit inside the 0-127 range - current one seems
to
be something like \xa3, and if I pass it through ord() function, it comes
out as character number 163.
Now issue is, yes, could just run through the hundreds of thousands of
characters in these resulting strings, and strip out any that are not
within the basic 0-127 range, but, that could result in corrupting data -
think so anyway.
Anyway, issue is, for example, if I try something like
str('\xa3').encode('utf-8') or str('\xa3').encode('ascii'), or
\xa3 already is a str; str(\xa3) is as redundant as
str(str(str(\xa3))) ;)
str('\xa3').encode('latin7') - that last one is actually our preferred
encoding for the MySQL database - they all just tell me they can't work
with a character out of range.
encode() goes from unicode to byte; you want to convert bytes to unicode
and
thus need decode().
In this context it is important that you tell us the Python version. In
Python 2 str.encode(encoding) is basically
str.decode(ascii).encode(encoding)
which is why you probably got a UnicodeDecodeError in the traceback:
\xa3.encode(latin7)
Traceback (most recent call last):
File stdin, line 1, in module
File /usr/lib/python2.7/encodings/iso8859_13.py, line 12, in encode
return codecs.charmap_encode(input,errors,encoding_table)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xa3 in position 0:
ordinal not in range(128)
\xa3.decode(latin7)
u'\xa3'
print \xa3.decode(latin7)
£
Aside: always include the traceback in your posts -- and always read it
carefully. The fact that latin7 is not mentioned might have given you a
hint that the problem was not what you thought it was.
Any thoughts on a sort of generic method/means to handle any/all
characters that might be out of range when having pulled them out of
something like these MS access databases?
Assuming the data in Access is not broken and that you know the encoding
decode() will work.
Another side note is for binary values that might store binary values, I
use something like the following to generate hex-based strings that work
alright when then inserting said same binary values into longblob fields,
but, don't think this would really help for what are really just most
likely badly chosen copy/pasted strings from documents, with strange
encoding, or something:
#sample code line for binary encoding into string output
s_values += 0x + str(l_data[J][I]).encode(hex).replace(\\, )
+
,
I would expect that you can feed bytestrings directly into blobs, without
any preparatory step. Try it, and if you get failures show us the failing
code and the corresponding traceback.
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: String character encoding when converting data from one type/format to another
Thanks.
Yes, using python 2.7, and all you said makes sense, so will check out the
talk, and the byte'ing, etc. (yes, bad joke, I know)
Issue with knowing encoding could just be that am pretty sure at least some
of the data capture is done via copy/paste from one MS app to another, which
could possibly result in a whole bunch of different character sets, etc.
being copied across, so it comes down to that while can't control sources of
data, need to manipulate/work with it to make it useful on our side now.
Thanks again
Jacob Kruger
Blind Biker
Skype: BlindZA
Roger Wilco wants to welcome you...to the space janitor's closet...
- Original Message -
From: Ned Batchelder n...@nedbatchelder.com
To: python-list@python.org
Sent: Wednesday, January 07, 2015 2:02 PM
Subject: Re: String character encoding when converting data from one
type/format to another
On 1/7/15 6:04 AM, Jacob Kruger wrote:
I'm busy using something like pyodbc to pull data out of MS access .mdb
files, and then generate .sql script files to execute against MySQL
databases using MySQLdb module, but, issue is forms of characters in
string values that don't fit inside the 0-127 range - current one seems
to be something like \xa3, and if I pass it through ord() function, it
comes out as character number 163.
Now issue is, yes, could just run through the hundreds of thousands of
characters in these resulting strings, and strip out any that are not
within the basic 0-127 range, but, that could result in corrupting data
- think so anyway.
That will definitely corrupt your data, since you will be discarding data.
Anyway, issue is, for example, if I try something like
str('\xa3').encode('utf-8') or str('\xa3').encode('ascii'), or
str('\xa3').encode('latin7') - that last one is actually our preferred
encoding for the MySQL database - they all just tell me they can't work
with a character out of range.
Are you using Python 2 or Python 3? This is one area where the two are
very different. I suspect you are on Python 2, in which case these all
fail the same way because you are calling encode on a bytestring. You
can't encode a bytestring, you can only encode a Unicode string, so encode
is helpfully trying to decode your bytestring first, using the default
encoding (ascii), and '\xa3' is not an ascii character.
If that was confusing, this talk covers these fundamentals:
http://bit.ly/unipain .
Any thoughts on a sort of generic method/means to handle any/all
characters that might be out of range when having pulled them out of
something like these MS access databases?
The best thing is to know what encoding was used to produce these byte
values. Then you can manipulate them as Unicode if you need to. The
second best thing is to simply pass them through as bytes.
Another side note is for binary values that might store binary values, I
use something like the following to generate hex-based strings that work
alright when then inserting said same binary values into longblob
fields, but, don't think this would really help for what are really just
most likely badly chosen copy/pasted strings from documents, with
strange encoding, or something:
#sample code line for binary encoding into string output
s_values += 0x + str(l_data[J][I]).encode(hex).replace(\\, )
+ ,
TIA
Jacob Kruger
Blind Biker
Skype: BlindZA
Roger Wilco wants to welcome you...to the space janitor's closet...
--
Ned Batchelder, http://nedbatchelder.com
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: String character encoding when converting data from one type/format to another
Thanks.
Yes, sorry didn't mention 2.7, and, unfortunately in this sense, all of this
will be running on windows machines.
Stay well
Jacob Kruger
Blind Biker
Skype: BlindZA
Roger Wilco wants to welcome you...to the space janitor's closet...
- Original Message -
From: Dave Angel da...@davea.name
To: python-list@python.org
Sent: Wednesday, January 07, 2015 2:22 PM
Subject: Re: String character encoding when converting data from one
type/format to another
On 01/07/2015 06:04 AM, Jacob Kruger wrote:
I'm busy using something like pyodbc to pull data out of MS access .mdb
files, and then generate .sql script files to execute
against MySQL databases using MySQLdb module, but, issue is forms of
characters in string values that don't fit inside
the 0-127 range - current one seems to be something like \xa3, and if I
pass it through ord() function,
it comes out as character number 163.
First question, of course is what version of Python. Clearly, you're not
using Python 3.x, so I'll assume 2.7. But you really should specify it in
your query.
Next question is what OS you're using. You're reading .mdb files, which
are most likely created in Windows, but that doesn't guarantee you're
actually using Windows to do this conversion.
Now issue is, yes, could just run through the hundreds of thousands of
characters in these resulting strings, and strip out any that are not
within the basic 0-127 range, but, that could result in corrupting data -
think so anyway.
Anyway, issue is, for example, if I try something like
str('\xa3').encode('utf-8') or str('\xa3').encode('ascii'), or
str('\xa3').encode('latin7') - that last one is actually our preferred
encoding for the MySQL database - they all just tell me they can't work
with a character out of range.
That's because your input data isn't ASCII. So before you encode it, you
have to decode it. Any idea what encoding it's already in? Maybe it's in
latin1, which permits all 256 values. Or utf-8, which permits a few
hundred thousand values, but uses multiple bytes for any of those over
127. Or any of hundreds of other encodings.
Does an .mdb file have a field specifying what encoding was used? Or do
you have some other external knowledge?
If you don't know what encoding it's currently in, you'll have to guess,
and the guess you're using so far is ASCII, which you know is false.
As for the encoding you should actually use in the database, that almost
certainly ought to be utf-8, which supports far more international
characters than latin1. And make sure the database has a way to tell the
future user what encoding you picked.
Any thoughts on a sort of generic method/means to handle any/all
characters that might be out of range when having pulled them out of
something like these MS access databases?
The only invalid characters are those which aren't valid in the encoding
used. Those can probably be safely converted to ? or something similar.
Another side note is for binary values that might store binary values, I
use something like the following to generate hex-based strings that work
alright when then inserting said same binary values into longblob fields,
but, don't think this would really help for what are really just most
likely badly chosen copy/pasted strings from documents, with strange
encoding, or something:
#sample code line for binary encoding into string output
s_values += 0x + str(l_data[J][I]).encode(hex).replace(\\, )
+ ,
Best to not pretend they're text at all. But if your db doesn't support
binary blobs, then use an encoding which supports all 256 values
unambiguously, while producing printable characters. Like uuencod, using
module uu
You might also look into mime, where you store the encoding of the data
with the data. See for example mimetypes.
--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: need some guidance on Python syntax smart editor for use with speech recognition
Eric, not really related, but, from the blind perspective, some of the end-users would also love to be able to define their own sets of voice commands, and have something like a personal assistant app handle that for them, instead of having to necessarily work with PC in normal ways all the time, and I played around with something relating to implementing the bare basics in a python command line app a little while ago, but, it only really used the windows voice recognition system/service, which meant it wasn't really useful from my side since the training interface for that is also pretty much useless to me, but anyway. Stay well Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... - Original Message - From: Eric S. Johansson e...@harvee.org To: python-list@python.org Sent: Monday, January 05, 2015 8:18 PM Subject: Re: need some guidance on Python syntax smart editor for use with speech recognition On 1/5/2015 7:24 AM, Chris Angelico wrote: On Mon, Jan 5, 2015 at 8:34 PM, Jonas Wielicki jo...@wielicki.name wrote: As a first iteration, I would try with any editor written in Python. Are you familiar with the ast[1] module? It could be worth trying to use this module and perform some kind of pattern matching on the results to recover the information. An AST parse throws away a lot of information. You can't easily reconstruct the original code from that; at best, you can decompile the AST back into functionally-equivalent source code, but it's not something you want to do as part of editing code. Chris is right. I looked at this and came up with the same conclusion. Here's a use case that I haven't talked about yet but is part of the problem of speech driven programming. I have a chunk of code that I created that is exactly right but it's in the wrong place or wrong method or even just the wrong environment. I need to capture it and then move it to the right place. At first glance, this would look like a simple cut-and-paste and, sometimes it is but more often it's not. Let's take the example of a simple file generator. Opening the file, iterating over the input until the input data transformation process runs out of data. This is a real common pattern and what changes are certain elements of the code, variables, strings whatever but also the indentation. So putting the old chunk of code into the new place requires transformation that most of you guys do by hand without thinking twice. Me it's, #()@!#$, and I use a whole bunch of kill words to strip out the old data and put in the new. Imagine being able to rip out a piece of code, identify the substitution points and the indentation adjustment levels and then being able to put in that piece of code in other places and just fill in the blanks as a part of the insertion dialogue. Now this gets us to the same point I was first talking about which is the ability to identify the idiom and code to operate on it as an idiom. some of the stuff I've seen with snippets is partly there, again it is a one-way transform. It seems to me that those of us with hands and those of us with broken hands could use this same functionality given the right supportive user interface. Now I've been living with this kind of idea for a while and I realize that may not make sense in this abstract written description. What do I need to fill in? However, back when I was doing my PEP 463 research, someone suggested looking into the 2to3 parser. I regretfully admit that I have yet to actually do so, but it ought in theory allow source-level transformations with a measure of intelligence. It's not a ready-made solution by any means, but it could be a useful tool. [2] Good idea. Hope this helps. I find that topic pretty interesting. Is there anything I can follow to see progress on this? Likewise! ChrisA Is there any problem with keeping the conversation going here or would you prefer some other way of talking about it? I've been hesitant to put my work up on github because it's flailing about with also the bloody loose ends because I'm trying to solve up number of problems all at the same time. The problem first being a framework where I can add speech driven UI elements to an editor so I can start experimenting with a bunch of these pieces. Another way you can help is be my hands. sometimes I just run out of hand time and it takes a while for me build up enough energy so I can spend the day working for money and a day working on this. --- eric -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Why For Loop Skips the Last element?
Not sure after quick skim/scan, but, I would generally use either for I in
range(len(inlist)): or just for it in inlist, instead of using while, etc.
Stay well
Jacob Kruger
Blind Biker
Skype: BlindZA
Roger Wilco wants to welcome you...to the space janitor's closet...
- Original Message -
From: Thuruv V
To: python-list@python.org
Sent: Wednesday, December 31, 2014 11:25 PM
Subject: Why For Loop Skips the Last element?
Hi all,
Here's the list. .
inlist = [Fossil Women's Natalie Stainless Steel Watch Brown (JR1385),
'Balmer Swiss Made Veyron Mens Watch: Black Band/ Black Dial (62625241)',
'Fortune NYC Ladies Quilted Dial Watch: Brown',
'Jeanne Collection w/ Swarovski Elements Watch: Dark Purple Band (62623659)',
'Swiss Legend Commander Chronograph: Watch Orange/Black (SL-10067-01-ORS)',
'Debussy Ladies Watch: Brown/Gray/Silver 14070 - 62625292',
'#2 Swiss Legend Commander Chronograph: Green (SL-10067-BB-017)',
'Auguste Jaccard 3 Timezone Watch: Black Band/ Gray Black Dial (62625184)'
]
My code :
i = 0
while i = len(inlist):
if 'watch' in str(inlist[i]).lower() or 'watches' in
str(inlist[i]).lower():
if 'women' in str(inlist[i]).lower() or 'female' in
str(inlist[i]).lower() or 'ladies' in str(inlist[i]).lower():
print 'FEMale Watch', inlist.index(i),str(i)
elif 'men' in str(inlist[i]).lower() or 'male' in
str(inlist[i]).lower() or 'chronograph' in str(inlist[i]).lower():
print 'Male Watch',inlist.index(i),str(i)
i = next(inlist)
When on executing python skips the last element('Auguste Jaccard 3 Timezone
Watch: Black Band/ Gray Black Dial (62625184)') in the list.
But when accessing through 'print inlist[0]' is possible,i.e it shows the
e;lement. but the list skips it. .
Please enlighten me. .
Thanks.
--
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: If One Line
One line assignment is ok, but, seems like you can't perform actions. #the following will work: I = 1 if True else 2 #but the following will generate an error: if I == 1: print(one) And, not sure if/how to work around that second one myself. Stay well Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... - Original Message - From: JC chalao.a...@gmail.com Newsgroups: comp.lang.python To: python-list@python.org Sent: Thursday, December 25, 2014 5:18 PM Subject: If One Line Hello, Is it possible in python: if ((x = a(b,c)) == 'TRUE'): print x Thanks. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: If One Line
One line assignment is ok, but, seems like you can't perform actions. #the following will work: I = 1 if True else 2 #but the following will generate an error: if I == 1: print(one) And, not sure if/how to work around that second one myself. Stay well Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... - Original Message - From: JC chalao.a...@gmail.com Newsgroups: comp.lang.python To: python-list@python.org Sent: Thursday, December 25, 2014 5:18 PM Subject: If One Line Hello, Is it possible in python: if ((x = a(b,c)) == 'TRUE'): print x Thanks. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: If One Line
Actually more that in the interpreter, it's prompting me with ... as if I had left out a closing ) or something, but, suppose it could work fine in an actual imported bit of code? Stay well Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... - Original Message - From: Skip Montanaro To: Jacob Kruger Cc: Python Sent: Thursday, December 25, 2014 6:26 PM Subject: Re: If One Line I don't get an error. I = 1 if True else 2 if I == 1: print(one) ... one What error did you get? Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: If One Line
Ok, makes sense - just slipped same one line if: action bit of code inside a function, and worked fine. Stay well Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... - Original Message - From: Skip Montanaro To: Jacob Kruger Cc: Python Sent: Thursday, December 25, 2014 7:45 PM Subject: Re: If One Line Actually more that in the interpreter, it's prompting me with ... as if I had left out a closing ) or something, but, suppose it could work fine in an actual imported bit of code? That's how it's supposed to work. Given that Python block structure is determined by indentation, you need some way to tell the interactive interpreter that the block is ended. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: If One Line
Ok, makes sense - just slipped same one line if: action bit of code inside a function, and worked fine. Stay well Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... - Original Message - From: Skip Montanaro To: Jacob Kruger Cc: Python Sent: Thursday, December 25, 2014 7:45 PM Subject: Re: If One Line Actually more that in the interpreter, it's prompting me with ... as if I had left out a closing ) or something, but, suppose it could work fine in an actual imported bit of code? That's how it's supposed to work. Given that Python block structure is determined by indentation, you need some way to tell the interactive interpreter that the block is ended. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: Simple background sound effect playback
Will have a look at it as well, but, also already tested generating a console/command line app using the pygame code to play an .ogg file, panning it left and right, and the compilation worked alright, and am now still busy reloading primary machine, so haven't taken it too much further, but let's see... Am not sure of redistribution licence either, and will need/want to test it on a couple of other types of machines. Stay well Jacob Kruger Blind Biker Skype: BlindZA Roger Wilco wants to welcome you...to the space janitor's closet... - Original Message - From: Akira Li 4kir4...@gmail.com To: python-list@python.org Sent: Monday, December 22, 2014 12:11 AM Subject: Re: Simple background sound effect playback Jacob Kruger ja...@blindza.co.za writes: Would prefer to use something free, that could work somewhat cross-platform, but, my primary target is for windows OS, and would primarily just want to be able to easily trigger playback of either .wav or .mp3 background sound effects, but, yes, would also be nice to be able to control them a little bit in terms of volume, possibly stereo panning as well as playback rate/frequency/pitch? I have used something called sound_lib, as well as another one relating to a sort of windows directSound effect, but, both of them had issues when working with either py2exe or cx_freeze when it came to compiling into executable, and main thing is would like to keep it simple...smile Suggestions? You could try GStreamer: it is free, cross-platform, it allows you to play media files in the background, to control volume, stereo panning (e.g., GstAudioPanorama), to change the playback rate while preserving pitch, it can use DirectShow on Windows, etc -- see http://gstreamer.freedesktop.org/features/ The downside is that it may be complex to install and use e.g., it probably works with py2exe but it won't be simple to configure. If you know other library that provides similar feature list while being less complex; do tell. -- Akira. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Simple background sound effect playback
Ok, thanks - the following seemed to work perfectly: #start code import pygame, time pygame.mixer.init(frequency=22050, size=-16, channels=2, buffer=4096) song = pygame.mixer.Sound(bird.ogg) channel = pygame.mixer.Sound.play(song) channel.set_volume(0, 1) time.sleep(1) channel.set_volume(1, 0) channel = pygame.mixer.Sound.play(song) time.sleep(1) #end code Will now just have to double-check/test compilation of something including something like this, using something like cx_freeze, etc., but, thanks again. Stay well Jacob Kruger Blind Biker Skype: BlindZA ...Roger Wilco wants to welcome you, to the space janitor's closet... ---original message--- catalinf...@gmail.com wrote in message news:e5f211da-d16a-4351-8094-55ba9d2d3...@googlegroups.com... See this : pygame.mixer.init(frequency=22050, size=-16, channels=2, buffer=4096) song = pygame.mixer.Sound(bird.ogg) pygame.mixer.Sound.play(song) and then use the channel.set_volume(0, 1) time.sleep(1) On Wednesday, December 17, 2014 10:02:30 PM UTC+2, Jacob Kruger wrote: Ok, trying simple code from a pygame tutorial snippet - nothing happens - just silent, but with no errors being returned: # import the pygame module import pygame # start pygame pygame.init() # load a sound file into memory sound = pygame.mixer.Sound(bird.ogg) channel = sound.play() -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Simple background sound effect playback
Ok, trying simple code from a pygame tutorial snippet - nothing happens - just silent, but with no errors being returned: # play a sound to the left, to the right and to the center # import the time standard module import time # import the pygame module import pygame # start pygame pygame.init() # load a sound file into memory sound = pygame.mixer.Sound(bird.ogg) # start playing the sound # and remember on which channel it is being played channel = sound.play() # set the volume of the channel # so the sound is only heard to the left channel.set_volume(1, 0) # wait for 1 second time.sleep(1) # do the same to the right channel = sound.play() channel.set_volume(0, 1) time.sleep(1) # do the same to the center channel = sound.play() channel.set_volume(1, 1) time.sleep(1) #end code Think this is why ignored pygame in past...smile - think it actually needs more than itself to be installed, or something. Jacob Kruger Blind Biker Skype: BlindZA ...Roger Wilco wants to welcome you, to the space janitor's closet... - Original Message - From: Ian Kelly To: Python Sent: Wednesday, December 17, 2014 3:49 AM Subject: Re: Simple background sound effect playback On Tue, Dec 16, 2014 at 9:57 AM, Jacob Kruger ja...@blindza.co.za wrote: Would prefer to use something free, that could work somewhat cross-platform, but, my primary target is for windows OS, and would primarily just want to be able to easily trigger playback of either .wav or .mp3 background sound effects, but, yes, would also be nice to be able to control them a little bit in terms of volume, possibly stereo panning as well as playback rate/frequency/pitch? I have used something called sound_lib, as well as another one relating to a sort of windows directSound effect, but, both of them had issues when working with either py2exe or cx_freeze when it came to compiling into executable, and main thing is would like to keep it simple...smile Suggestions? https://wiki.python.org/moin/Audio/ -- -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Simple background sound effect playback
Would prefer to use something free, that could work somewhat cross-platform, but, my primary target is for windows OS, and would primarily just want to be able to easily trigger playback of either .wav or .mp3 background sound effects, but, yes, would also be nice to be able to control them a little bit in terms of volume, possibly stereo panning as well as playback rate/frequency/pitch? I have used something called sound_lib, as well as another one relating to a sort of windows directSound effect, but, both of them had issues when working with either py2exe or cx_freeze when it came to compiling into executable, and main thing is would like to keep it simple...smile Suggestions? TIA Jacob Kruger Blind Biker Skype: BlindZA ...Roger Wilco wants to welcome you, to the space janitor's closet... -- https://mail.python.org/mailman/listinfo/python-list
New member - 'odd' question
Firstly, python is one of my two current focus areas, and while I still work with python 2.7, sort of for backward compatibility, my other focus area is web development, making use of PHP, javascript, jQuery, MySQL, etc. And, I develop on windows7 64 bit machines, in conjunction with screen reader software - either jaws, or NVDA, if that means much...smile Now, one of the current pieces of work am busy with involves sort of automating the process of converting some MS access databases to MySQL, and manipulating/rearranging the data during the conversion process. I am implementing this using a python app, making use of wxPython to implement the GUI, and am using pyodbc to pull in the data structures, and read the data from .mdb files, and then am trying to use either pyodbc, or MySQLdb modules to then handle interaction with the MySQL side of things, and the MySQL database engine is part of the WAMP installation on my dev machine, FWIW. Now, on my primary, desktop machine, it seems like if I implement pyodbc connections and cursors to handle the input data/structures from an .mdb file, and then write out the relevant SQL queries to text files, and then try to read them in, split them into separate statements, and execute them against the MySQL database, that part of it runs fine, but, the odd issue seems to be that if I did in fact first create/open a pyodbc connection object, along with it's relevant cursor object, and even if make sure close both of them, then on my one machine, after then initiating the direct MySQLdb connection object, I am still unable to actually close the whole app down - the main class is a sort of inheritor of wx.Frame, and before closing it down, I destroy both the Frame and all it's children, but, the main oddity is that while this works fine on my other laptop, on my desktop machine it always just ends up telling me python.exe is not responding, and would I like to force it to close? And, it's definitely got something to do with the combination of having created and/or destroyed the pyodbc objects first, and then creating MySQLdb objects, since either half works fine by itself, but not together - on one machine anyway. Just wondering what can be causing this to occur on one machine, but not the other, relatively similar, but maybe cleaner machine, and how to try track down something like this? That's also why am posting this here - trying to figure out how to track down what's causing this odd behaviour. TIA Jacob Kruger Blind Biker Skype: BlindZA ...Roger Wilco wants to welcome you, to the space janitor's closet... -- https://mail.python.org/mailman/listinfo/python-list
