Re: GPG wrapper, ECC 25519 compatible?
On 03.09.19 05:28, rmli...@riseup.net wrote:
> But I just don't understand how to get
> and pass information back to the gpg command line prompts at all, not to
> mention automating the process.
The manpage describes how to enable the machine-parsable interface,
which is exactly what you want. Then:
import subprocess
inputs = [
"11",
"Q",
"1",
"0",
"y",
"username",
"usern...@user.net",
"none",
"O",
]
input_data = ("\n".join(inputs) + "\n").encode()
subprocess.check_output([ "gpg2", "--expert", "--full-gen-key",
"--with-colons", "--command-fd", "0", "--status-fd", "1" ], input =
input_data)
Cheers,
Joe
--
"Performance ist nicht das Problem, es läuft ja nachher beides auf der
selben Hardware." -- Hans-Peter Diettrich in d.s.e.
--
https://mail.python.org/mailman/listinfo/python-list
Announcing Scm Workbench 0.9.3 for Git, Mercurial and Subversion
SCM Workbench features • Support Subversion (svn), Mercurial (hg) and Git projects. • Experimental support for Perforce (P4) • Easy to learn and use • Builtin User Guide describes the operation and features of the application. • Add project wizard can scan for all your existing projects. • All subversion client operations in a GUI • Many Git client operations in a GUI • GUI git rebase • Some mercurial (hg) client operations in a GUI • Enhanced operations (subversion rename of modified files etc) • Support for Dark mode • Support software development workflow • Builtin GUI diff showing line and character diffs • Ability to diff between revisions in a files history • Runs on Windows, Mac OS X and Unix platforms Please visit http://scm-workbench.barrys-emacs.org/ for downloads, git source, user guide and further information on SCM Workbench. New in 0.9.3 • Lots of improvement since the last release • Update to use python3.7, PyQt5 5.12 and pysvn with svn 1.12 Barry -- https://mail.python.org/mailman/listinfo/python-list
Re: Problem while integrating unittest with setuptools
Finally I found why my setup.py dosen't work. I didn't put a `__init__.py` in my `test` folder, thus Python dosen't think it's a package. That's why it found the wrong package. Thank you. On Mon, Sep 02, 2019 at 04:28:50PM +0200, dieter wrote: > YuXuan Dong writes: > > I have uninstalled `six` using `pip uninstall six` but the problem is still > > there. > > Your traceback shows that `six` does not cause your problem. > It is quite obvious that a `test_winreg` will want to load the > `wingreg` module. > > > As you suggested, I have checked the traceback and found the exception is > > caused by > > `/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/test/test_winreg.py` > > on my machine with homebrew-installed Python3. > > > > I have realized that the problem may be caused of `test-suite=test` in my > > `setup.py`. `setuptools` will find the `test` package. But the package it > > found is not in my project. It found > > `/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/test`. > > Yes. Apparently, you run the Python test suite - and depending on > platform and available infrastructure some of its tests fail. > > In my package `dm.xmlsec.binding` I use > `test_suite='dm.xmlsec.binding.tests.testsuite'`. > Maybe, you can ensure to get the correct test suite in a similar > way for your package. > > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Execute complex shell commands within python and obtain the output.
On Tue, 03 Sep 2019 08:24:17 +1000, Cameron Simpson wrote:
> It seems reasonable to me, but I would not use stderr=subprocess.STDOUT.
> Why did you do that? The stderr stream pretty much exists to avoid
> polluting stdout with error messages.
Thanks for your suggestion.
>
> For really complex stuff you're often better writing a shell script and
> invoking that script, and avoiding shell=True. This has the advantages
> that:
>
> (a) it avoids shell=True, a common source of accidents
>
> (b) you don't have to embed shell punctuation in a Python string, which
> itself has punctuation
>
> (c) you can run the script yourself by hand for testing purposes,
> outside the Python programme
>
> (d) you're not restricted to the shell; the script might be in awk or
> any number of other languages
>
> Finally, the .decode('utf8') assumes your locale is UTF8 based. It
> probably is, but if it isn't then you may get mojibake.
Nowadays, most of the os use utf8 as the default locale. Am I wrong?
--
https://mail.python.org/mailman/listinfo/python-list
GPG wrapper, ECC 25519 compatible?
I'm looking for a non-gui GPG wrapper that supports elliptic-curve 25519 for development. The only one's I've seen that support ECC, only support the p-XYZ curves created by NIST. I've tried monkey-patching python-gnupg to add 25519 support, but I can't get my head around that codebase. Or if you have solid advice as to how I'd go about writing my own, that'd be wonderful. Since, I've also tried many versions of creating a new wrapper package. But I just don't understand how to get and pass information back to the gpg command line prompts at all, not to mention automating the process. On linux, you can see the command line prompts I'm trying to work with by running: >gpg2 --expert --full-gen-key The default responses I'd like to give are: >11 >Q >1 >0 >y >username >usern...@user.net >none >O Any tips that don't include using a different curve? -- https://mail.python.org/mailman/listinfo/python-list
Re: Append some stuff into a file with only the last appended line reserved.
On 01Sep2019 04:13, Hongyi Zhao wrote: I want to append some log of pycurl's downloading info to file, and I only want to reserve the last appended line when write. How to do this? Please describe this in more detail. Present a little pycurl output and then explain what portion of it should land in the log file. Cheers, Cameron Simpson -- https://mail.python.org/mailman/listinfo/python-list
Re: PYTHON DIDNT DETECTED
On 31Aug2019 21:30, АРТЁМ БОЗАДЖИ wrote: Traceback (most recent call last): File "", line 1, in NameError: name 'python' is not defined and more You seem to have left off the command which gave this error message. C:\Users\VeNoMD>python -v [..."python" from your command prompt works...] I would guess that you hare put the command "python" _inside_ a python script. The command "python" is for use _outside_ the script, to invoke the script. We'd need to see what you did, and your script, to offer better advice. Cheers, Cameron Simpson -- https://mail.python.org/mailman/listinfo/python-list
Re: Execute complex shell commands within python and obtain the output.
On 02Sep2019 13:20, Hongyi Zhao wrote:
I try to execute some complex shell commands with in python and obtain
the output, I tried the following method:
For python 3.x:
import subprocess
cmd= some_complex_command_with_pipe_and_others
ps = subprocess.Popen
(cmd,shell=True,stdout=subprocess.PIPE,stderr=subprocess.STDOUT)
output = ps.communicate()[0].decode('utf8')
Is this the correct usage for this case?
It seems reasonable to me, but I would not use stderr=subprocess.STDOUT.
Why did you do that? The stderr stream pretty much exists to avoid
polluting stdout with error messages.
For really complex stuff you're often better writing a shell script and
invoking that script, and avoiding shell=True. This has the advantages
that:
(a) it avoids shell=True, a common source of accidents
(b) you don't have to embed shell punctuation in a Python string, which
itself has punctuation
(c) you can run the script yourself by hand for testing purposes,
outside the Python programme
(d) you're not restricted to the shell; the script might be in awk or
any number of other languages
Finally, the .decode('utf8') assumes your locale is UTF8 based. It
probably is, but if it isn't then you may get mojibake.
Cheers,
Cameron Simpson
--
https://mail.python.org/mailman/listinfo/python-list
Re: "Edit With Python" option missing
On 8/31/19, Akash verma wrote: > "Edit With Python" option missing from message context when right clicked > with mouse . If you mean "Edit with IDLE", this means that you've changed the file association. It's no longer using the default "Python.File" program identifier (progid). Create an empty .py script on your desktop. Right click it, and select "Open with" -> "Choose another app". Select the "Python" app that has the Python logo with a rocket on it. At the bottom of the dialog, select the option to "Always use this app to open .py files". Click "OK". This can happen if some other program grabbed the file association -- perhaps automatically, or perhaps you approved the change and don't remember it. It can also be the case that a user mistakenly tries to fix this by selecting the "Look for an app on this PC" option in the "Open with" dialog, which auto-generates a progid for the file association. (1) You probably do not want an auto-generated progid because you'll lose installed features of the application. In the case of Python, that's the "Edit with IDLE" menu and the drop-handler shell extension that enables dragging and dropping files on a script in Explorer. Also, if you end up selecting an installed "python.exe" instead of the "py.exe" launcher, you'll lose shebang support in scripts. (2) Windows creates a progid that's intended for opening a data file. It doesn't have `*%` in the template, so passing command-line arguments won't work. That said, if you do end up associating .py files with an auto-generated progid, at least now you know a simple way to switch back to the original progid. -- https://mail.python.org/mailman/listinfo/python-list
Re: Issue About Install pyinstaller
On 9/1/19, Mehmet Furkan ÇOLAK wrote: > > I did “cmd > pip install pyinstaller > enter” > > from C:\Users\Furkan ÇOLAK\AppData\Local\Programs\Python\Python37-32 > but there isnt Script folder. > > İn cmd I see this kind of ERROR > 'pip' is not recognized as an internal or external command, > operable program or batch file. CMD failed to find a `pip` command when it searched the directories in the PATH environment variable for "pip" plus each file extension in the PATHEXT environment variable. To locate all of the "pip.exe" files in the "%LocalAppData%\Programs" tree, use dir /b /s "%LocalAppData%\Programs\pip.exe" or where.exe /r "%LocalAppData%\Programs" pip.exe Then either run pip.exe using its fully-qualified path, or temporarily add the parent directory to PATH. For example: set PATH=%PATH%;%LocalAppData%\Programs\Python\Python37-32\Scripts You can permanently modify PATH using the system environment-variable editor (e.g. via control panel -> system -> advanced system settings -> environment variables). Note that directories in PATH should never be quoted. Also, Python's installer has an advanced option to add the scripts directory to PATH (i.e. "add Python to environment variables"). You can rerun the installer and modify the installation to select this option. -- https://mail.python.org/mailman/listinfo/python-list
Re: Hi how do I import files inside a txt file?
On 3/09/19 1:48 AM, Spencer Du wrote:
On Monday, 2 September 2019 15:29:07 UTC+2, Joel Goldstick wrote:
On Mon, Sep 2, 2019 at 9:21 AM Spencer Du wrote:
On Monday, 2 September 2019 15:03:52 UTC+2, Joel Goldstick wrote:
On Mon, Sep 2, 2019 at 8:46 AM Spencer Du wrote:
On Monday, 2 September 2019 13:36:06 UTC+2, Pankaj Jangid wrote:
Spencer Du writes:
How do i import files inside a txt file if they exist in the current directory?
Here is the current code but I dont know how to do it correctly.
import paho.mqtt.client as mqtt
from mqtt import *
import importlib
import os
import os.path
# from stateMachine import *
with open("list_of_devices.txt", "r") as reader:
for item in reader:
try:
os.getcwd()
print("hi")
except:
print("error")
This is "list_of_devices.txt":
test1,test2
Each name refers to a python file.
My interpretation is that you want to read a file (list_of_devices.txt)
and this file contains names of other files and you want to read those
files as well and do something with them (read or print or whatever).
You can approach it like this: write a function to read a file and work
on it. Like this,
def fn(fname):
with open(fname, "r") as f:
try:
# work with f
except:
print("error")
Then use this function in your code that you have writen. Like this
with open("list_of_devices.txt", "r") as reader:
for item in reader:
try:
fn(item)
except:
print("error")
In the example that you gave, you have written contents of
"list_of_devices.txt" as
test1,test2
Take care to read them as comma separated. Or if you have control then
write them on separate lines.
Regards.
--
Pankaj Jangid
Hi Pankaj
I dont understand so what is complete code then?
Thanks
Spencer
--
https://mail.python.org/mailman/listinfo/python-list
Pardon me for guessing, but your question seems to imply that you know
how you want to do something .. but I'm not sure you have tackled your
problem correctly.
My guess is: Depending upon the names listed in a text file, you want
to do different imports into your program. You don't yet know how to
read a file with python.
First, when you run your program, python compiles it in order. Since
you don't know what you want to import until after you run your
program, you can't import those modules. You may be able to run a
program to read the module list, then have it output to a new file the
code you eventually want to run based on the modules you discovered.
That sounds cute in a way, but probably not in a good way. You could
also surround import statements with try/except code that will import
what it can, and alert you when it can't
Can you give us the bigger picture of what you want to accomplish?
This might lead to a better solution than the one you are thinking of
now
--
Joel Goldstick
http://joelgoldstick.com/blog
http://cc-baseballstats.info/stats/birthdays
Hi
I have a txt file which contains the names of files. They are .py files. I want
to import them into a python file if they exists in current directory and if
the name of file does not exist then print out error and not import. How do I
do this?
Thanks
Spencer
Here is a discussion on Stack overflow that lays out how you can
dynamically import files. This should get you started in the right
direction. First, see if you can write code to read the file, and
retrieve the names of the modules you want to import. Come back if
you stumble with your code for that
https://stackoverflow.com/questions/301134/how-to-import-a-module-given-its-name-as-string
--
https://mail.python.org/mailman/listinfo/python-list
--
Joel Goldstick
http://joelgoldstick.com/blog
http://cc-baseballstats.info/stats/birthdays
Ok I have this code to retrieve the names of modules I want to import. Now how
do I check if they exist in current directory and if they exist import them
into the python program. Thanks.
with open("list_of_devices.txt", "r") as f:
for item in f:
print(item)
Perhaps it is time to slow-down and take a breather?
The answer to this step has already been covered in this thread!
Many languages require one to *anticipate* every contingency - anything
that could go 'wrong'. For example, that a file does not exist and
cannot be imported.
However, Python follows a philosophy that it is "easier to ask
forgiveness than it is to get permission"*. In other words, to use the
try...except construct - in this case, to attempt an import (the "try")
and if that fails (probably because the file does not exist) then to
defend/react accordingly (the "except" - "except" = "exception").
In other words, only worry about 'the problem' (post fact), should it
arise. This works quite neatly for your use-case.
Such is what is called "a Python idiom" - the way things ar
Help needed to run some code!!!!
Hi
How can I execute "from devicesEmbedded import *" after this: "print("Device
added to list")" in GUI.py because currently if I have the import added at the
top of GUI.py file it always executes first before the GUI.py file is executed.
I want the devicesEmbedded.py to execute after GUI.py has executed first.
Thanks
Spencer
GUI.py
import logging
from datetime import timedelta
import time
from thespian.actors import *
from transitions import Machine
import paho.mqtt.client as mqtt
import importlib
import os
import os.path
import sys
from PyQt5.QtWidgets import *
from PyQt5.QtCore import *
from PyQt5 import QtWidgets, uic
from mqtt import *
# from devicesEmbedded import *
import json
class MainWindow(QtWidgets.QMainWindow):
def __init__(self,parent = None):
QMainWindow.__init__(self)
super(MainWindow, self).__init__(parent)
self.mdi = QMdiArea()
self.setCentralWidget(self.mdi)
self.setMinimumSize(QSize(800, 600))
self.setWindowTitle("PyQt button example -
pythonprogramminglanguage.com")
pybutton = QPushButton('Add device', self)
pybutton.clicked.connect(self.importbutton)
pybutton.move(100, 400)
pybutton.resize(150, 32)
self.textbox = QLineEdit(self)
self.textbox.move(100,350)
self.textbox.resize(100, 32)
self.fileName_UI = ""
def importbutton(self):
self.fileName_UI = self.textbox.text()
self.loadGUI()
def getGUIFilename(self):
return self.fileName_UI
def loadGUI(self):
print("Searching file", self.fileName_UI)
try:
module = __import__(self.fileName_UI)
my_class = getattr(module, "SubWindow")
sub = QMdiSubWindow()
sub.setWidget(my_class())
sub.setWindowTitle("New GUI: " + self.fileName_UI)
self.mdi.addSubWindow(sub)
sub.show()
print("creating new instance " + self.fileName_UI)
client = device("Device")
client.run()
client.loop_start() # start the loop
# device_message = self.fileName_UI
time.sleep(2)
print("Subscribing to topic",
"microscope/light_sheet_microscope/UI")
client.subscribe("microscope/light_sheet_microscope/UI")
print("Publishing message to topic",
"microscope/light_sheet_microscope/UI/list_of_devices")
client.publish("microscope/light_sheet_microscope/UI/list_of_devices",
json.dumps({"type": "device", "payload":{"name": self.fileName_UI, "cmd":
"adding device"}}, indent=2))
time.sleep(1) # wait
client.loop_stop() # stop the loop
print("Device added" + "\n")
client.run()
client.loop_start()
time.sleep(2)
print("Subscribing to topic",
"microscope/light_sheet_microscope/UI/list_of_devices")
client.subscribe("microscope/light_sheet_microscope/UI/list_of_devices")
print("Publishing message to topic",
"microscope/light_sheet_microscope/UI/list_of_devices")
client.publish("microscope/light_sheet_microscope/UI/list_of_devices",
self.fileName_UI + " added to device list")
time.sleep(1)
client.loop_stop()
listofdevices = []
listofdevices.append(self.fileName_UI)
with open("list_of_devices.txt", "a+") as myfile:
for item in listofdevices:
myfile.write(item + ",")
print(item)
print(listofdevices)
print("Device added to list")
except:
print("creating new instance " + self.fileName_UI)
client = device("Device")
client.run()
client.loop_start() # start the loop
device_message = self.fileName_UI
time.sleep(2)
print("Subscribing to topic",
"microscope/light_sheet_microscope/UI")
client.subscribe("microscope/light_sheet_microscope/UI")
print("Publishing message to topic",
"microscope/light_sheet_microscope/UI")
client.publish("microscope/light_sheet_microscope/UI",
json.dumps({"type": "device", "payload":{"name": self.fileName_UI}}, indent=2))
time.sleep(2) # wait
client.loop_stop() # stop the loop
print(device_message + ".py " + "file doesn't exist")
print("Device not added")
if __name__ == "__main__":
app = QApplication(sys.argv)
mainWin = MainWindow()
a = mainWin.show()
try:
mainWin.show()
os.remove("list_of_devices.txt")
print("Awaiting devices to be launched")
except:
print("Awaiting devices to be launched")
publishedMessage = mainWin.getGUIFilename()
Help needed urgently for running some code!!!!
Hi
I want to execute
"from devicesEmbedded import *": in GUI.py after all code in GUI.py is run.
Also how do I make the devicesEmbedded.py reload and run when a txt file is
created in the name of "list_of_devices.txt" in the GUI.py python program.
GUI.py
import logging
from datetime import timedelta
import time
from thespian.actors import *
from transitions import Machine
import paho.mqtt.client as mqtt
import importlib
import os
import os.path
import sys
from PyQt5.QtWidgets import *
from PyQt5.QtCore import *
from PyQt5 import QtWidgets, uic
from mqtt import *
# from devicesEmbedded import *
import json
class MainWindow(QtWidgets.QMainWindow):
def __init__(self,parent = None):
QMainWindow.__init__(self)
super(MainWindow, self).__init__(parent)
self.mdi = QMdiArea()
self.setCentralWidget(self.mdi)
self.setMinimumSize(QSize(800, 600))
self.setWindowTitle("PyQt button example -
pythonprogramminglanguage.com")
pybutton = QPushButton('Add device', self)
pybutton.clicked.connect(self.importbutton)
pybutton.move(100, 400)
pybutton.resize(150, 32)
self.textbox = QLineEdit(self)
self.textbox.move(100,350)
self.textbox.resize(100, 32)
self.fileName_UI = ""
def importbutton(self):
self.fileName_UI = self.textbox.text()
self.loadGUI()
def getGUIFilename(self):
return self.fileName_UI
def loadGUI(self):
print("Searching file", self.fileName_UI)
try:
module = __import__(self.fileName_UI)
my_class = getattr(module, "SubWindow")
sub = QMdiSubWindow()
sub.setWidget(my_class())
sub.setWindowTitle("New GUI: " + self.fileName_UI)
self.mdi.addSubWindow(sub)
sub.show()
print("creating new instance " + self.fileName_UI)
client = device("Device")
client.run()
client.loop_start() # start the loop
# device_message = self.fileName_UI
time.sleep(2)
print("Subscribing to topic",
"microscope/light_sheet_microscope/UI")
client.subscribe("microscope/light_sheet_microscope/UI")
print("Publishing message to topic",
"microscope/light_sheet_microscope/UI/list_of_devices")
client.publish("microscope/light_sheet_microscope/UI/list_of_devices",
json.dumps({"type": "device", "payload":{"name": self.fileName_UI, "cmd":
"adding device"}}, indent=2))
time.sleep(1) # wait
client.loop_stop() # stop the loop
print("Device added" + "\n")
client.run()
client.loop_start()
time.sleep(2)
print("Subscribing to topic",
"microscope/light_sheet_microscope/UI/list_of_devices")
client.subscribe("microscope/light_sheet_microscope/UI/list_of_devices")
print("Publishing message to topic",
"microscope/light_sheet_microscope/UI/list_of_devices")
client.publish("microscope/light_sheet_microscope/UI/list_of_devices",
self.fileName_UI + " added to device list")
time.sleep(1)
client.loop_stop()
listofdevices = []
listofdevices.append(self.fileName_UI)
with open("list_of_devices.txt", "a+") as myfile:
for item in listofdevices:
myfile.write(item + ",")
print(item)
print(listofdevices)
print("Device added to list")
except:
print("creating new instance " + self.fileName_UI)
client = device("Device")
client.run()
client.loop_start() # start the loop
device_message = self.fileName_UI
time.sleep(2)
print("Subscribing to topic",
"microscope/light_sheet_microscope/UI")
client.subscribe("microscope/light_sheet_microscope/UI")
print("Publishing message to topic",
"microscope/light_sheet_microscope/UI")
client.publish("microscope/light_sheet_microscope/UI",
json.dumps({"type": "device", "payload":{"name": self.fileName_UI}}, indent=2))
time.sleep(2) # wait
client.loop_stop() # stop the loop
print(device_message + ".py " + "file doesn't exist")
print("Device not added")
if __name__ == "__main__":
app = QApplication(sys.argv)
mainWin = MainWindow()
a = mainWin.show()
try:
mainWin.show()
os.remove("list_of_devices.txt")
print("Awaiting devices to be launched")
except:
print("Awaiting devices to be launched")
publishedMessage = mainWin.getGUIFilename()
sys.exit(app.exec_())
devicesEmbedded.py:
import random
import asyncio
from actorio import Actor, Message, DataMessage, ask, EndMainLoop, Reference
from mqtt import *
# from GUI im
Re: Problem while integrating unittest with setuptools
YuXuan Dong writes: > I have uninstalled `six` using `pip uninstall six` but the problem is still > there. Your traceback shows that `six` does not cause your problem. It is quite obvious that a `test_winreg` will want to load the `wingreg` module. > As you suggested, I have checked the traceback and found the exception is > caused by > `/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/test/test_winreg.py` > on my machine with homebrew-installed Python3. > > I have realized that the problem may be caused of `test-suite=test` in my > `setup.py`. `setuptools` will find the `test` package. But the package it > found is not in my project. It found > `/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/test`. Yes. Apparently, you run the Python test suite - and depending on platform and available infrastructure some of its tests fail. In my package `dm.xmlsec.binding` I use `test_suite='dm.xmlsec.binding.tests.testsuite'`. Maybe, you can ensure to get the correct test suite in a similar way for your package. -- https://mail.python.org/mailman/listinfo/python-list
Re: Hi how do I import files inside a txt file?
On Monday, 2 September 2019 15:29:07 UTC+2, Joel Goldstick wrote:
> On Mon, Sep 2, 2019 at 9:21 AM Spencer Du wrote:
> >
> > On Monday, 2 September 2019 15:03:52 UTC+2, Joel Goldstick wrote:
> > > On Mon, Sep 2, 2019 at 8:46 AM Spencer Du wrote:
> > > >
> > > > On Monday, 2 September 2019 13:36:06 UTC+2, Pankaj Jangid wrote:
> > > > > Spencer Du writes:
> > > > >
> > > > > > How do i import files inside a txt file if they exist in the
> > > > > > current directory?
> > > > > >
> > > > > > Here is the current code but I dont know how to do it correctly.
> > > > > >
> > > > > > import paho.mqtt.client as mqtt
> > > > > > from mqtt import *
> > > > > > import importlib
> > > > > > import os
> > > > > > import os.path
> > > > > > # from stateMachine import *
> > > > > >
> > > > > > with open("list_of_devices.txt", "r") as reader:
> > > > > > for item in reader:
> > > > > > try:
> > > > > > os.getcwd()
> > > > > > print("hi")
> > > > > > except:
> > > > > > print("error")
> > > > > >
> > > > > > This is "list_of_devices.txt":
> > > > > > test1,test2
> > > > > >
> > > > > > Each name refers to a python file.
> > > > > >
> > > > > My interpretation is that you want to read a file
> > > > > (list_of_devices.txt)
> > > > > and this file contains names of other files and you want to read those
> > > > > files as well and do something with them (read or print or whatever).
> > > > >
> > > > > You can approach it like this: write a function to read a file and
> > > > > work
> > > > > on it. Like this,
> > > > >
> > > > > def fn(fname):
> > > > > with open(fname, "r") as f:
> > > > > try:
> > > > > # work with f
> > > > > except:
> > > > > print("error")
> > > > >
> > > > > Then use this function in your code that you have writen. Like this
> > > > >
> > > > > with open("list_of_devices.txt", "r") as reader:
> > > > > for item in reader:
> > > > > try:
> > > > > fn(item)
> > > > > except:
> > > > > print("error")
> > > > >
> > > > > In the example that you gave, you have written contents of
> > > > > "list_of_devices.txt" as
> > > > >
> > > > > test1,test2
> > > > >
> > > > > Take care to read them as comma separated. Or if you have control then
> > > > > write them on separate lines.
> > > > >
> > > > > Regards.
> > > > > --
> > > > > Pankaj Jangid
> > > >
> > > > Hi Pankaj
> > > >
> > > > I dont understand so what is complete code then?
> > > >
> > > > Thanks
> > > > Spencer
> > > > --
> > > > https://mail.python.org/mailman/listinfo/python-list
> > >
> > > Pardon me for guessing, but your question seems to imply that you know
> > > how you want to do something .. but I'm not sure you have tackled your
> > > problem correctly.
> > >
> > > My guess is: Depending upon the names listed in a text file, you want
> > > to do different imports into your program. You don't yet know how to
> > > read a file with python.
> > >
> > > First, when you run your program, python compiles it in order. Since
> > > you don't know what you want to import until after you run your
> > > program, you can't import those modules. You may be able to run a
> > > program to read the module list, then have it output to a new file the
> > > code you eventually want to run based on the modules you discovered.
> > > That sounds cute in a way, but probably not in a good way. You could
> > > also surround import statements with try/except code that will import
> > > what it can, and alert you when it can't
> > >
> > > Can you give us the bigger picture of what you want to accomplish?
> > > This might lead to a better solution than the one you are thinking of
> > > now
> > >
> > > --
> > > Joel Goldstick
> > > http://joelgoldstick.com/blog
> > > http://cc-baseballstats.info/stats/birthdays
> >
> > Hi
> >
> > I have a txt file which contains the names of files. They are .py files. I
> > want to import them into a python file if they exists in current directory
> > and if the name of file does not exist then print out error and not import.
> > How do I do this?
> >
> > Thanks
> > Spencer
>
> Here is a discussion on Stack overflow that lays out how you can
> dynamically import files. This should get you started in the right
> direction. First, see if you can write code to read the file, and
> retrieve the names of the modules you want to import. Come back if
> you stumble with your code for that
>
> https://stackoverflow.com/questions/301134/how-to-import-a-module-given-its-name-as-string
> > --
> > https://mail.python.org/mailman/listinfo/python-list
>
>
>
> --
> Joel Goldstick
> http://joelgoldstick.com/blog
> http://cc-baseballstats.info/stats/birthdays
Ok I have this code to retrieve the names of modules I want to import. Now how
do I check if they exist in current directory and if they exist import them
into the python progra
Re: Hi how do I import files inside a txt file?
On Mon, Sep 2, 2019 at 9:21 AM Spencer Du wrote:
>
> On Monday, 2 September 2019 15:03:52 UTC+2, Joel Goldstick wrote:
> > On Mon, Sep 2, 2019 at 8:46 AM Spencer Du wrote:
> > >
> > > On Monday, 2 September 2019 13:36:06 UTC+2, Pankaj Jangid wrote:
> > > > Spencer Du writes:
> > > >
> > > > > How do i import files inside a txt file if they exist in the current
> > > > > directory?
> > > > >
> > > > > Here is the current code but I dont know how to do it correctly.
> > > > >
> > > > > import paho.mqtt.client as mqtt
> > > > > from mqtt import *
> > > > > import importlib
> > > > > import os
> > > > > import os.path
> > > > > # from stateMachine import *
> > > > >
> > > > > with open("list_of_devices.txt", "r") as reader:
> > > > > for item in reader:
> > > > > try:
> > > > > os.getcwd()
> > > > > print("hi")
> > > > > except:
> > > > > print("error")
> > > > >
> > > > > This is "list_of_devices.txt":
> > > > > test1,test2
> > > > >
> > > > > Each name refers to a python file.
> > > > >
> > > > My interpretation is that you want to read a file (list_of_devices.txt)
> > > > and this file contains names of other files and you want to read those
> > > > files as well and do something with them (read or print or whatever).
> > > >
> > > > You can approach it like this: write a function to read a file and work
> > > > on it. Like this,
> > > >
> > > > def fn(fname):
> > > > with open(fname, "r") as f:
> > > > try:
> > > > # work with f
> > > > except:
> > > > print("error")
> > > >
> > > > Then use this function in your code that you have writen. Like this
> > > >
> > > > with open("list_of_devices.txt", "r") as reader:
> > > > for item in reader:
> > > > try:
> > > > fn(item)
> > > > except:
> > > > print("error")
> > > >
> > > > In the example that you gave, you have written contents of
> > > > "list_of_devices.txt" as
> > > >
> > > > test1,test2
> > > >
> > > > Take care to read them as comma separated. Or if you have control then
> > > > write them on separate lines.
> > > >
> > > > Regards.
> > > > --
> > > > Pankaj Jangid
> > >
> > > Hi Pankaj
> > >
> > > I dont understand so what is complete code then?
> > >
> > > Thanks
> > > Spencer
> > > --
> > > https://mail.python.org/mailman/listinfo/python-list
> >
> > Pardon me for guessing, but your question seems to imply that you know
> > how you want to do something .. but I'm not sure you have tackled your
> > problem correctly.
> >
> > My guess is: Depending upon the names listed in a text file, you want
> > to do different imports into your program. You don't yet know how to
> > read a file with python.
> >
> > First, when you run your program, python compiles it in order. Since
> > you don't know what you want to import until after you run your
> > program, you can't import those modules. You may be able to run a
> > program to read the module list, then have it output to a new file the
> > code you eventually want to run based on the modules you discovered.
> > That sounds cute in a way, but probably not in a good way. You could
> > also surround import statements with try/except code that will import
> > what it can, and alert you when it can't
> >
> > Can you give us the bigger picture of what you want to accomplish?
> > This might lead to a better solution than the one you are thinking of
> > now
> >
> > --
> > Joel Goldstick
> > http://joelgoldstick.com/blog
> > http://cc-baseballstats.info/stats/birthdays
>
> Hi
>
> I have a txt file which contains the names of files. They are .py files. I
> want to import them into a python file if they exists in current directory
> and if the name of file does not exist then print out error and not import.
> How do I do this?
>
> Thanks
> Spencer
Here is a discussion on Stack overflow that lays out how you can
dynamically import files. This should get you started in the right
direction. First, see if you can write code to read the file, and
retrieve the names of the modules you want to import. Come back if
you stumble with your code for that
https://stackoverflow.com/questions/301134/how-to-import-a-module-given-its-name-as-string
> --
> https://mail.python.org/mailman/listinfo/python-list
--
Joel Goldstick
http://joelgoldstick.com/blog
http://cc-baseballstats.info/stats/birthdays
--
https://mail.python.org/mailman/listinfo/python-list
Execute complex shell commands within python and obtain the output.
Hi,
I try to execute some complex shell commands with in python and obtain
the output, I tried the following method:
For python 3.x:
import subprocess
cmd= some_complex_command_with_pipe_and_others
ps = subprocess.Popen
(cmd,shell=True,stdout=subprocess.PIPE,stderr=subprocess.STDOUT)
output = ps.communicate()[0].decode('utf8')
Is this the correct usage for this case?
Regards
--
https://mail.python.org/mailman/listinfo/python-list
Re: Hi how do I import files inside a txt file?
On Monday, 2 September 2019 15:03:52 UTC+2, Joel Goldstick wrote:
> On Mon, Sep 2, 2019 at 8:46 AM Spencer Du wrote:
> >
> > On Monday, 2 September 2019 13:36:06 UTC+2, Pankaj Jangid wrote:
> > > Spencer Du writes:
> > >
> > > > How do i import files inside a txt file if they exist in the current
> > > > directory?
> > > >
> > > > Here is the current code but I dont know how to do it correctly.
> > > >
> > > > import paho.mqtt.client as mqtt
> > > > from mqtt import *
> > > > import importlib
> > > > import os
> > > > import os.path
> > > > # from stateMachine import *
> > > >
> > > > with open("list_of_devices.txt", "r") as reader:
> > > > for item in reader:
> > > > try:
> > > > os.getcwd()
> > > > print("hi")
> > > > except:
> > > > print("error")
> > > >
> > > > This is "list_of_devices.txt":
> > > > test1,test2
> > > >
> > > > Each name refers to a python file.
> > > >
> > > My interpretation is that you want to read a file (list_of_devices.txt)
> > > and this file contains names of other files and you want to read those
> > > files as well and do something with them (read or print or whatever).
> > >
> > > You can approach it like this: write a function to read a file and work
> > > on it. Like this,
> > >
> > > def fn(fname):
> > > with open(fname, "r") as f:
> > > try:
> > > # work with f
> > > except:
> > > print("error")
> > >
> > > Then use this function in your code that you have writen. Like this
> > >
> > > with open("list_of_devices.txt", "r") as reader:
> > > for item in reader:
> > > try:
> > > fn(item)
> > > except:
> > > print("error")
> > >
> > > In the example that you gave, you have written contents of
> > > "list_of_devices.txt" as
> > >
> > > test1,test2
> > >
> > > Take care to read them as comma separated. Or if you have control then
> > > write them on separate lines.
> > >
> > > Regards.
> > > --
> > > Pankaj Jangid
> >
> > Hi Pankaj
> >
> > I dont understand so what is complete code then?
> >
> > Thanks
> > Spencer
> > --
> > https://mail.python.org/mailman/listinfo/python-list
>
> Pardon me for guessing, but your question seems to imply that you know
> how you want to do something .. but I'm not sure you have tackled your
> problem correctly.
>
> My guess is: Depending upon the names listed in a text file, you want
> to do different imports into your program. You don't yet know how to
> read a file with python.
>
> First, when you run your program, python compiles it in order. Since
> you don't know what you want to import until after you run your
> program, you can't import those modules. You may be able to run a
> program to read the module list, then have it output to a new file the
> code you eventually want to run based on the modules you discovered.
> That sounds cute in a way, but probably not in a good way. You could
> also surround import statements with try/except code that will import
> what it can, and alert you when it can't
>
> Can you give us the bigger picture of what you want to accomplish?
> This might lead to a better solution than the one you are thinking of
> now
>
> --
> Joel Goldstick
> http://joelgoldstick.com/blog
> http://cc-baseballstats.info/stats/birthdays
Hi
I have a txt file which contains the names of files. They are .py files. I want
to import them into a python file if they exists in current directory and if
the name of file does not exist then print out error and not import. How do I
do this?
Thanks
Spencer
--
https://mail.python.org/mailman/listinfo/python-list
Re: Hi how do I import files inside a txt file?
On Mon, Sep 2, 2019 at 8:46 AM Spencer Du wrote:
>
> On Monday, 2 September 2019 13:36:06 UTC+2, Pankaj Jangid wrote:
> > Spencer Du writes:
> >
> > > How do i import files inside a txt file if they exist in the current
> > > directory?
> > >
> > > Here is the current code but I dont know how to do it correctly.
> > >
> > > import paho.mqtt.client as mqtt
> > > from mqtt import *
> > > import importlib
> > > import os
> > > import os.path
> > > # from stateMachine import *
> > >
> > > with open("list_of_devices.txt", "r") as reader:
> > > for item in reader:
> > > try:
> > > os.getcwd()
> > > print("hi")
> > > except:
> > > print("error")
> > >
> > > This is "list_of_devices.txt":
> > > test1,test2
> > >
> > > Each name refers to a python file.
> > >
> > My interpretation is that you want to read a file (list_of_devices.txt)
> > and this file contains names of other files and you want to read those
> > files as well and do something with them (read or print or whatever).
> >
> > You can approach it like this: write a function to read a file and work
> > on it. Like this,
> >
> > def fn(fname):
> > with open(fname, "r") as f:
> > try:
> > # work with f
> > except:
> > print("error")
> >
> > Then use this function in your code that you have writen. Like this
> >
> > with open("list_of_devices.txt", "r") as reader:
> > for item in reader:
> > try:
> > fn(item)
> > except:
> > print("error")
> >
> > In the example that you gave, you have written contents of
> > "list_of_devices.txt" as
> >
> > test1,test2
> >
> > Take care to read them as comma separated. Or if you have control then
> > write them on separate lines.
> >
> > Regards.
> > --
> > Pankaj Jangid
>
> Hi Pankaj
>
> I dont understand so what is complete code then?
>
> Thanks
> Spencer
> --
> https://mail.python.org/mailman/listinfo/python-list
Pardon me for guessing, but your question seems to imply that you know
how you want to do something .. but I'm not sure you have tackled your
problem correctly.
My guess is: Depending upon the names listed in a text file, you want
to do different imports into your program. You don't yet know how to
read a file with python.
First, when you run your program, python compiles it in order. Since
you don't know what you want to import until after you run your
program, you can't import those modules. You may be able to run a
program to read the module list, then have it output to a new file the
code you eventually want to run based on the modules you discovered.
That sounds cute in a way, but probably not in a good way. You could
also surround import statements with try/except code that will import
what it can, and alert you when it can't
Can you give us the bigger picture of what you want to accomplish?
This might lead to a better solution than the one you are thinking of
now
--
Joel Goldstick
http://joelgoldstick.com/blog
http://cc-baseballstats.info/stats/birthdays
--
https://mail.python.org/mailman/listinfo/python-list
Re: Hi how do I import files inside a txt file?
On Monday, 2 September 2019 13:36:06 UTC+2, Pankaj Jangid wrote:
> Spencer Du writes:
>
> > How do i import files inside a txt file if they exist in the current
> > directory?
> >
> > Here is the current code but I dont know how to do it correctly.
> >
> > import paho.mqtt.client as mqtt
> > from mqtt import *
> > import importlib
> > import os
> > import os.path
> > # from stateMachine import *
> >
> > with open("list_of_devices.txt", "r") as reader:
> > for item in reader:
> > try:
> > os.getcwd()
> > print("hi")
> > except:
> > print("error")
> >
> > This is "list_of_devices.txt":
> > test1,test2
> >
> > Each name refers to a python file.
> >
> My interpretation is that you want to read a file (list_of_devices.txt)
> and this file contains names of other files and you want to read those
> files as well and do something with them (read or print or whatever).
>
> You can approach it like this: write a function to read a file and work
> on it. Like this,
>
> def fn(fname):
> with open(fname, "r") as f:
> try:
> # work with f
> except:
> print("error")
>
> Then use this function in your code that you have writen. Like this
>
> with open("list_of_devices.txt", "r") as reader:
> for item in reader:
> try:
> fn(item)
> except:
> print("error")
>
> In the example that you gave, you have written contents of
> "list_of_devices.txt" as
>
> test1,test2
>
> Take care to read them as comma separated. Or if you have control then
> write them on separate lines.
>
> Regards.
> --
> Pankaj Jangid
Hi Pankaj
I dont understand so what is complete code then?
Thanks
Spencer
--
https://mail.python.org/mailman/listinfo/python-list
Re: Hi how do I import files inside a txt file?
On Monday, 2 September 2019 13:36:06 UTC+2, Pankaj Jangid wrote:
> Spencer Du writes:
>
> > How do i import files inside a txt file if they exist in the current
> > directory?
> >
> > Here is the current code but I dont know how to do it correctly.
> >
> > import paho.mqtt.client as mqtt
> > from mqtt import *
> > import importlib
> > import os
> > import os.path
> > # from stateMachine import *
> >
> > with open("list_of_devices.txt", "r") as reader:
> > for item in reader:
> > try:
> > os.getcwd()
> > print("hi")
> > except:
> > print("error")
> >
> > This is "list_of_devices.txt":
> > test1,test2
> >
> > Each name refers to a python file.
> >
> My interpretation is that you want to read a file (list_of_devices.txt)
> and this file contains names of other files and you want to read those
> files as well and do something with them (read or print or whatever).
>
> You can approach it like this: write a function to read a file and work
> on it. Like this,
>
> def fn(fname):
> with open(fname, "r") as f:
> try:
> # work with f
> except:
> print("error")
>
> Then use this function in your code that you have writen. Like this
>
> with open("list_of_devices.txt", "r") as reader:
> for item in reader:
> try:
> fn(item)
> except:
> print("error")
>
> In the example that you gave, you have written contents of
> "list_of_devices.txt" as
>
> test1,test2
>
> Take care to read them as comma separated. Or if you have control then
> write them on separate lines.
>
> Regards.
> --
> Pankaj Jangid
Hi I dont really understand this. So what would be the complete code? Also I
want to import files if it exists based on what is in the txt file.
Thanks
Spencer
--
https://mail.python.org/mailman/listinfo/python-list
Re: Hi how do I import files inside a txt file?
Spencer Du writes:
> How do i import files inside a txt file if they exist in the current
> directory?
>
> Here is the current code but I dont know how to do it correctly.
>
> import paho.mqtt.client as mqtt
> from mqtt import *
> import importlib
> import os
> import os.path
> # from stateMachine import *
>
> with open("list_of_devices.txt", "r") as reader:
> for item in reader:
> try:
> os.getcwd()
> print("hi")
> except:
> print("error")
>
> This is "list_of_devices.txt":
> test1,test2
>
> Each name refers to a python file.
>
My interpretation is that you want to read a file (list_of_devices.txt)
and this file contains names of other files and you want to read those
files as well and do something with them (read or print or whatever).
You can approach it like this: write a function to read a file and work
on it. Like this,
def fn(fname):
with open(fname, "r") as f:
try:
# work with f
except:
print("error")
Then use this function in your code that you have writen. Like this
with open("list_of_devices.txt", "r") as reader:
for item in reader:
try:
fn(item)
except:
print("error")
In the example that you gave, you have written contents of
"list_of_devices.txt" as
test1,test2
Take care to read them as comma separated. Or if you have control then
write them on separate lines.
Regards.
--
Pankaj Jangid
--
https://mail.python.org/mailman/listinfo/python-list
Hi how do I import files inside a txt file?
Hi
How do i import files inside a txt file if they exist in the current directory?
Here is the current code but I dont know how to do it correctly.
import paho.mqtt.client as mqtt
from mqtt import *
import importlib
import os
import os.path
# from stateMachine import *
with open("list_of_devices.txt", "r") as reader:
for item in reader:
try:
os.getcwd()
print("hi")
except:
print("error")
This is "list_of_devices.txt":
test1,test2
Each name refers to a python file.
Thanks
Spencer
--
https://mail.python.org/mailman/listinfo/python-list
Re: Problem while integrating unittest with setuptools
Thank you. It helps. I have uninstalled `six` using `pip uninstall six` but the problem is still there. As you suggested, I have checked the traceback and found the exception is caused by `/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/test/test_winreg.py` on my machine with homebrew-installed Python3. I have realized that the problem may be caused of `test-suite=test` in my `setup.py`. `setuptools` will find the `test` package. But the package it found is not in my project. It found `/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/test`. To verify my guess, I renamed the `test` folder in the above folder. As expected, another exception is raised. The problem now is that, how could I config my `setup.py` to work properly? On 2019/9/2, 13:07, "Python-list on behalf of dieter" wrote: YuXuan Dong writes: > I met a problem while I ran `python setup.py test`: > > unittest.case.SkipTest: No module named 'winreg' > ... no windows modules should be necessary ... I know apparently unexplainable "no module named ..." messages as a side effect of the use of "six". "six" is used to facilitate the development of components usable for both Python 2 and Python 3. Among others, it contains the module "six.moves" which uses advance Python features to allow the import of Python modules from different locations in the package hierarchy and also handles name changes. This can confuse other components using introspection (they see modules in "six.move" which are not really available). To find out if something like this is the cause of your problem, please try to get a traceback. It might be necessary to use a different test runner for this (I like much "zope.testrunner" and my wrapper "dm.zopepatches.ztest"). -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Need help: integrating unittest with setuptools
No, it doesn't. The stackoverflow question you posted is about the renaming of `winreg`. `_winreg` is renamed to `winreg`. That's why the poster can't find the module. My program is written for and running on unix-like systems. I think `winreg` should not appear here. I have tried running `pip3 install winreg` on MacOS and I got: `Could not find a version that satisfies the requirement winreg`. On 2019/9/2, 08:11, "Python-list on behalf of Sayth Renshaw" wrote: On Monday, 2 September 2019 04:44:29 UTC+10, YuXuan Dong wrote: > Hi, everybody: > > I have met a problem while I ran `python setup.py test`: > > unittest.case.SkipTest: No module named 'winreg' > > I ran the command in MacOS and my project is written for only UNIX-like systems. I don't use any Windows-specified API. How dose `winreg` come here? > > In my `setup.py`: > > test_suite="test" > > In my `test/test.py`: > > import unittest > > class TestAll(unittest.TestCase): > def testall(self): > return None > > It works if I ran `python -m uniittest test.py` alone but raises the above exception if I ran `python setup.py test`. > > I'm working on this for the whole day, searching for every keywords I can think of with Google but can't find why or how. Could you help me? Thanks. > > -- > YX. D. Does this help? https://stackoverflow.com/questions/4320761/importerror-no-module-named-winreg-python3 Sayth -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
EuroPython 2019: Please send in your feedback
EuroPython 2019 is over now and so it’s time to ask around for what we can improve next year. If you attended EuroPython 2019, please take a few moments and fill in our feedback form, if you haven’t already done so: EuroPython 2019 Feedback Form * https://www.europython-society.org/feedback-2019 * We will leave the feedback form online for a few weeks and then use the information as basis for the work on EuroPython 2020 and also intend to post a summary of the multiple choice questions (not the comments to protect your privacy) on our website. Many thanks in advance. Help spread the word Please help us spread this message by sharing it on your social networks as widely as possible. Thank you ! Link to the blog post: https://blog.europython.eu/post/187438617892/europython-2019-please-send-in-your-feedback Tweet: https://twitter.com/europython/status/1168464170716717057 Enjoy, -- EuroPython 2019 Team https://ep2019.europython.eu/ https://www.europython-society.org/ -- https://mail.python.org/mailman/listinfo/python-list
Re: a,b = 2,3 and [a,b] = [2,3]
Am Montag, 2. September 2019 00:49:05 UTC+2 schrieb Hongyi Zhao: > Hi, > > What's differences: > > a,b = 2,3 and [a,b] = [2,3] > > Regards In this example the result is the same but the second one builds, internally, an additional list, therefore isn't as sufficient as the first one. -- https://mail.python.org/mailman/listinfo/python-list
Re: Using exec with embedded python interpreter 3.7
@MRAB, I'm building a notepad++ plugin which can execute the written code and if one writes help(os) it gets executed via exec(editor.getText()) and output redirected to the plugin console window. Sorry to you as well as I have also replied to you directly. Thank you -- https://mail.python.org/mailman/listinfo/python-list
a,b = 2,3 and [a,b] = [2,3]
Hi, What's differences: a,b = 2,3 and [a,b] = [2,3] Regards -- https://mail.python.org/mailman/listinfo/python-list
Re: Using exec with embedded python interpreter 3.7
Just saw, that I replied to you directly instead to python list, sorry. That did it, changed encoding from function to property and now I'm able to call help(object) Thank you. -- https://mail.python.org/mailman/listinfo/python-list
Re: a,b = 2,3 and [a,b] = [2,3]
On Mon, Sep 2, 2019 at 6:31 PM Alan Bawden wrote:
>
> Dang! There is exactly the instruction sequence I just argued could be
> optimized away still sitting right there. So maybe my belief that this is
> being done by peephole optimization is in fact incorrect? So I went and
> tried again:
>
> bash-4.2$ python3 -E
> Python 3.6.6 (default, Aug 13 2018, 18:24:23)
> [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> def f():
> ... a, b = 2, 3
> ... a, b = [2, 3]
> ... a, b = b, a
> ... a, b = [b, a]
> ...
> >>> import dis
> >>> dis.dis(f)
> 2 0 LOAD_CONST 3 ((2, 3))
> 2 UNPACK_SEQUENCE 2
> 4 STORE_FAST 0 (a)
> 6 STORE_FAST 1 (b)
>
> 3 8 LOAD_CONST 1 (2)
> 10 LOAD_CONST 2 (3)
> 12 ROT_TWO
> 14 STORE_FAST 0 (a)
> 16 STORE_FAST 1 (b)
>
> 4 18 LOAD_FAST1 (b)
> 20 LOAD_FAST0 (a)
> 22 ROT_TWO
> 24 STORE_FAST 0 (a)
> 26 STORE_FAST 1 (b)
>
> 5 28 LOAD_FAST1 (b)
> 30 LOAD_FAST0 (a)
> 32 ROT_TWO
> 34 STORE_FAST 0 (a)
> 36 STORE_FAST 1 (b)
> 38 LOAD_CONST 0 (None)
> 40 RETURN_VALUE
>
> OK, now I'm confused. How come I'm not seeing the
> BUILD_LIST/UNPACK_SEQUENCE sequence that you're seeing?
>
> > This is with CPython 3.9. It's entirely possible that other Pythons
> > and/or other versions of CPython may give different results, but with
> > this particular interpreter, the list is not optimized away.
>
> I actually did try this with several different versions of CPython going
> back to 2.4 and up to 3.6, and they all behave this way for me. Maybe
> something changed after 3.6? Weird...
This is indeed fascinating. Something DID indeed change. I tried this
slightly shorter version in a few different Pythons:
def f():
a, b = b, a
a, b = [b, a]
import dis, sys
print(sys.version)
dis.dis(f)
# Show the code identically on 2.x and 3.x
print(repr(f.__code__.co_code).lstrip("b"))
Here's what I learned:
CPython 2.7: ROT_TWO, bytecode
CPython 3.4: ROT_TWO, bytecode
CPython 3.5: ROT_TWO, bytecode
CPython 3.6: ROT_TWO, wordcode
CPython 3.7: UNPACK_SEQUENCE
CPython 3.8: UNPACK_SEQUENCE
CPython 3.9: UNPACK_SEQUENCE
PyPy 5.6 (2.7): out-of-order LOAD/STORE
PyPyJS (2.7.9): out-of-order LOAD/STORE
Jython 2.5.3: unable to disassemble
MicroPython 3.4: no 'dis' module or __code__ attr
Brython: unable to disassemble
CPython 3.6 made the change to wordcode. If you have a 3.5 hanging
around, you should be able to see this easily in the disassembly,
because the LOAD_FAST operations require three bytes each in 3.5, but
only one (two byte) operation in 3.6, but the ROT_TWO is a single byte
in 3.5 and now requires two in 3.6. PyPy can reorder operations
knowing that it won't affect anything, and thus optimizes it down to
"load b, load a, store b, store a" regardless of the syntax.
But the curious difference happens in 3.7. I don't know what changed
to cause this, but from there on, the list gets built and then
unpacked.
This may represent a performance regression. Alternatively, just take
it as a recommendation to always do your variable exchanges WITHOUT
square brackets, and you'll be fine.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: a,b = 2,3 and [a,b] = [2,3]
Chris Angelico writes: > On Mon, Sep 2, 2019 at 12:36 PM Alan Bawden wrote: ... > > > > a,b = 2,3 and [a,b] = [2,3] ... > > It looks to me like they generate identical code. The first one calls the > > construction of a tuple, where the second one calls for the construction of > > a list. It would be surprising if the compiler optimized the tuple > > away, but failed to optimize the list away! > > > > Well, you can find out with the 'dis' module. Actually, when I wrote "It looks to me", I was in fact looking at the results of calling `dis.dis'! But I had unconciously changed what the OP wrote to make it more realistic. I tested: >>> def f(x, y): x, y = y, x [x, y] = [y, x] >>> dis.dis(f) 2 0 LOAD_FAST1 (y) 3 LOAD_FAST0 (x) 6 ROT_TWO 7 STORE_FAST 0 (x) 10 STORE_FAST 1 (y) 3 13 LOAD_FAST1 (y) 16 LOAD_FAST0 (x) 19 ROT_TWO 20 STORE_FAST 0 (x) 23 STORE_FAST 1 (y) 26 LOAD_CONST 0 (None) 29 RETURN_VALUE And I observed that whatever peephole optimization got rid of the tuple also got rid of the list. Clearly "BUILD_LIST 2" or "BUILD_TUPLE 2" followed by "UNPACK_SEQUENCE 2" can be replaced by "ROT_TWO", and the compiler knew that! It never occured to me that the case where all the elements of the sequence to the right of the "=" were constants would change that, but yes, you are right about that (but read on...): > >>> def f(): > ... a, b = 2, 3 > ... a, b = [2, 3] > ... > >>> dis.dis(f) > 2 0 LOAD_CONST 1 ((2, 3)) > 2 UNPACK_SEQUENCE 2 > 4 STORE_FAST 0 (a) > 6 STORE_FAST 1 (b) And indeed, if they are all constants, then the tuple itself is a constant, and the obvious peephole optimization is no longer available! But wait... > 3 8 LOAD_CONST 2 (2) > 10 LOAD_CONST 3 (3) > 12 BUILD_LIST 2 > 14 UNPACK_SEQUENCE 2 > 16 STORE_FAST 0 (a) > 18 STORE_FAST 1 (b) Dang! There is exactly the instruction sequence I just argued could be optimized away still sitting right there. So maybe my belief that this is being done by peephole optimization is in fact incorrect? So I went and tried again: bash-4.2$ python3 -E Python 3.6.6 (default, Aug 13 2018, 18:24:23) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> def f(): ... a, b = 2, 3 ... a, b = [2, 3] ... a, b = b, a ... a, b = [b, a] ... >>> import dis >>> dis.dis(f) 2 0 LOAD_CONST 3 ((2, 3)) 2 UNPACK_SEQUENCE 2 4 STORE_FAST 0 (a) 6 STORE_FAST 1 (b) 3 8 LOAD_CONST 1 (2) 10 LOAD_CONST 2 (3) 12 ROT_TWO 14 STORE_FAST 0 (a) 16 STORE_FAST 1 (b) 4 18 LOAD_FAST1 (b) 20 LOAD_FAST0 (a) 22 ROT_TWO 24 STORE_FAST 0 (a) 26 STORE_FAST 1 (b) 5 28 LOAD_FAST1 (b) 30 LOAD_FAST0 (a) 32 ROT_TWO 34 STORE_FAST 0 (a) 36 STORE_FAST 1 (b) 38 LOAD_CONST 0 (None) 40 RETURN_VALUE OK, now I'm confused. How come I'm not seeing the BUILD_LIST/UNPACK_SEQUENCE sequence that you're seeing? > This is with CPython 3.9. It's entirely possible that other Pythons > and/or other versions of CPython may give different results, but with > this particular interpreter, the list is not optimized away. I actually did try this with several different versions of CPython going back to 2.4 and up to 3.6, and they all behave this way for me. Maybe something changed after 3.6? Weird... -- Alan Bawden -- https://mail.python.org/mailman/listinfo/python-list
