urllib2 redirect error
Hi, I've tried versions of the following but continue to get errors: - snip - url = 'https://www.usps.com/send/official-abbreviations.htm' request = urllib2.build_opener(urllib2.HTTPRedirectHandler).open(url) - snip - Generates an exception: urllib2.HTTPError: HTTP Error 301: The HTTP server returned a redirect error that would lead to an infinite loop. The last 30x error message was: Moved Permanently The following using the requests library hangs: - snip - response = requests.get(url) - snip - I'm fresh out of ideas and any suggestions you may have would be greatly appreciated. Thanks in advance, Sumit -- https://mail.python.org/mailman/listinfo/python-list
urllib2 redirect error
Hi, I've tried various versions but continue to get the following error: -- https://mail.python.org/mailman/listinfo/python-list
Re: ANN: binario - simple work with binary files
On Tuesday, September 2, 2014 6:05:19 AM UTC+5:30, Tim Roberts wrote:
> Rustom Mody wrote:
> >On Tuesday, August 26, 2014 6:58:42 AM UTC+5:30, Tim Roberts wrote:
> >> To the equivalent code with struct:
> >> import struct
> >> dscrp = "H?fs5B"
> >> f = open('file.dat')
> >> stuff = struct.unpack( dscrp, f.read() )
> >> print stuff
> >> In both cases, you have to KNOW the format of the data beforehand. If you
> >> do a read_short where you happen to have written a float, disaster ensues.
> >> I don't really see that you've added very much.
> >I thought much the same.
> >However notice your f.read(). Its type is string.
> >What if file.dat is a 1GB wav file?
> f.seek(512000)
> stuff = struct.unpack( dscrp, f.read(128) )
And what if the struct you are (trying to) unpack is greater or less
than 128 bytes?
> The point is that files already know how to position themselves.
Sure.
But its not enough. One can write a generator that yields one char
at a time. How to feed that to struct.unpack??
--
https://mail.python.org/mailman/listinfo/python-list
Re: error while writing program to send mail.
On Tue, 02 Sep 2014 05:05:41 +0530, Om Prakash wrote:
> fp = open("message", 'rb')
"message" here is a string literal
> fp.close
should be fp.close()
> msg['Subject'] = 'The contents of $s' % message
message here is a variable. The variable named message has not previously
had a value assigned to it.
$s should be %s
If you want the subject to be the filename, then what you need to do is:
1) assign the filename to some variable
2) open the file using the variable that contains the filename
3) use the variable that contains the filename to generate the subject
eg (assuming suitable imports etc) and the message is in a text file
called message.txt:
msgfile = "message.txt"
fp = open( msgfile, "r" )
msg = MIMEText(fp.read())
fp.close()
msg['Subject'] = 'The contents of %s' % msgfile
--
Denis McMahon, denismfmcma...@gmail.com
--
https://mail.python.org/mailman/listinfo/python-list
Re: Editing text with an external editor in Python
Cameron Simpson wrote: > It is not just about being hacked. > > It is about being robust in the face of unusual setups. > > If I were producing this function for general use (even my own personal > general use) it would need to be reliable. That includes things like > $TMPDIR having spaces in it (or other unfortunate punctuation). Ah, gotcha. That makes sense. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Define proxy in windows 7
On Tue, Sep 2, 2014 at 12:06 PM, Cameron Simpson wrote: > I am not a Windows user, but on UNIX systems the format of http_proxy and > https_proxy is: > > http://proxyname:3128/ > > being the proxy hostname and port number respectively. You're saying: > > proxyname:8080 > > instead. (Note, https_proxy _also_ starts with "http:", not "https:" because > the communication with the proxy is HTTP.) > > Try the longer form. And ensure you have the port number right; proxies do > not normally listen on port 80; they tend to listen on port 3128 or 8080. These tips may help. (Though on Windows, where port 80 requires no special privileges, it's more likely for a proxy to use that than it is under Unix. So it's entirely possible it is actually on 80.) But what I'm seeing is a problem with environment variable setting in the first place - or else a transcription problem in the original post. Try this: set http_proxy=proxy name:80 If that doesn't work,*copy and paste* what you're doing and what happens when you do. Include the prompt, the command, and its output. That'll make it easier for us to figure out what's going on. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Define proxy in windows 7
On 02Sep2014 06:25, Om Prakash wrote: I am wondering how to define proxy setting in env variable on windows 7, I want this so i can use pip to pull packages for me, the same setting though working earlier on windows xp. http_proxy = "proxy name:80" now this same setting doesn't work, i tried doing in the cmd.exe prompt. set http_proxy "proxy name:80" P.S. i am a normal user and don't have admin privleges. I am not a Windows user, but on UNIX systems the format of http_proxy and https_proxy is: http://proxyname:3128/ being the proxy hostname and port number respectively. You're saying: proxyname:8080 instead. (Note, https_proxy _also_ starts with "http:", not "https:" because the communication with the proxy is HTTP.) Try the longer form. And ensure you have the port number right; proxies do not normally listen on port 80; they tend to listen on port 3128 or 8080. Cheers, -- To understand recursion, you must first understand recursion. -- https://mail.python.org/mailman/listinfo/python-list
Re: Editing text with an external editor in Python
On Monday, September 1, 2014 11:11:34 AM UTC-5, Steven D'Aprano wrote:
> Python's input() or raw_input() function is good for getting a single line
>
> of text from the user. But what if you want a more substantial chunk of
>
> text from the user? Here's how to call out to an external editor such as
>
> ed, nano, vim, emacs, and even GUI text editors:
>
>
>
> import tempfile
>
>
>
> def edit(editor, content=''):
>
> f = tempfile.NamedTemporaryFile(mode='w+')
>
> if content:
>
> f.write(content)
>
> f.flush()
>
> command = editor + " " + f.name
>
> status = os.system(command)
>
> f.seek(0, 0)
>
> text = f.read()
>
> f.close()
>
> assert not os.path.exists(f.name)
>
> return (status, text)
>
>
>
>
>
> Anyone able to test it on Windows for me please?
>
>
>
>
>
> More here:
>
>
>
> https://code.activestate.com/recipes/578926/
>
>
>
>
>
> --
>
> Steven
here's a full blown text editor (GUI) that gets a "substantial chunk of text"
from the user. It's almost finished but is perfectly functional right now.
Two files:
main file:
#!/usr/bin/env python
import os
import sys
import glob
import webbrowser
from GtkApp import *
from GPYedit_conf import Preferences
APPLICATION_NAME = 'GPYedit'
class GPYedit(GtkApp_Toplevel):
# For each tab in the notebook, we will store
# our data representing each file (or empty buffer)
# in a list. Each item is a dictionary keeping track of the:
# - Python file object
# - Three components of editing area: scrolled window, textview, and
buffer (per tab)
# - Full pathname of the file being edited
# - Text shown in the notebook widget tabs
open_files = [ ]
# Keep track of which buffer we're dealing with.
# Each time the notebook page is switched, this number
# will change (see 'on_nb_page_switched' callback). This value
# is used as the index into the open files list to get at the
# file-specific information and widgets.
current_tab = 0
# User preferences will be accessible through this attribute.
# The Preferences class will be initialized from directives
# found in the gpyedit_settings.ini file.
preferences = Preferences()
def __init__(this):
"""
This is where it all starts. Begin by setting
the window geometry and title and decide whether
to create a new empty file or use the arguments provided.
"""
GtkApp_Toplevel.__init__(this)
this.window.set_title("GPYedit")
(width, height) = GPYedit.preferences.get_window_dimensions()
this.window.set_default_size(width, height)
this.build_GUI()
if len(sys.argv) > 1:
names = sys.argv[1:]
for name in names:
if os.path.exists(name) and os.path.isfile(name):
this.tab_new_from_contents(name)
else:
print 'File "' + name + '" doesn\'t exist.'
else:
this.create_new_file()
def build_GUI(this):
"""
Create the main interface components.
These are
- vbox: Main vertical box for laying out widgets
- menu_bar: self explanatory
- notebook: The tabbed container holding our file buffers
"""
this.vbox = gtk.VBox(False, 0)
this.menu_bar = gtk.MenuBar()
this.notebook = gtk.Notebook()
this.notebook.set_scrollable(True)
this.notebook.connect("switch-page", this.on_nb_page_switch)
this.create_menus()
this.create_toolbar()
this.vbox.pack_start(this.notebook, True, True, 0)
this.window.add(this.vbox)
def create_new_file(this, menu_item = None):
"""
Create a blank buffer with no associated Python file object or name
(yet)
NOTE: menu_item is a parameter here because
this method will be used as a signal handler
also for the File menu 'new' item and the prototype
for that handler requires this parameter. It is not used though.
"""
(scrolled_win, textview, buf) = this.editing_area_new()
label = gtk.Label("Untitled")
edit_area = { 'scrolled_window': scrolled_win, 'textview':
textview, 'buffer': buf }
# Store everything we know
GPYedit.open_files.append({'file_object': None,
'edit_area': edit_area,
'filename':None,
'label': label})
index = this.notebook.append_page(scrolled_win, label)
this.notebook.show_all()
return GPY
Re: error while writing program to send mail.
On 09/02/2014 05:29 AM, MRAB wrote:
On 2014-09-02 00:35, Om Prakash wrote:
Hi,
I am writing this program from
https://docs.python.org/2/library/email-examples.html
but getting the error as
singhom@debian:~/pythons$ python send_email.py
Traceback (most recent call last):
File "send_email.py", line 18, in
msg['Subject'] = 'The contents of $s' % message
NameError: name 'message' is not defined
i know the error would be something simple and i am overlooking it, any
help would be highly appreciated, I am sorry, but I am very new to
python programming.
code starts here.
#/usr/bin/python2.7 -tt
## sending a simple text message using python.
import smtplib
from email.mime.text import MIMEText
# Open file for reading.
fp = open("message", 'rb')
# Create a plain text message.
msg = MIMEText(fp.read())
fp.close
That should be:
fp.close()
me = "torque.in...@gmail.com"
you = "oomprak...@gmail.com"
msg['Subject'] = 'The contents of $s' % message
You're trying to use the format operator, but:
1. You never bound the name 'message' to a value, hence:
NameError: name 'message' is not defined
2. The format string contains '$s' instead of '%s'.
msg['From'] = me
msg['To'] = you
# Send message thorugh localhost but don't include the envelope headers.
s = smtplib.SMTP('localhost')
s.sendmail(me, [you], msg.as_string())
s.quit()
Thanks a lot. will fix this and my overlooking of things too. :)
--
https://mail.python.org/mailman/listinfo/python-list
Define proxy in windows 7
Hi, I am wondering how to define proxy setting in env variable on windows 7, I want this so i can use pip to pull packages for me, the same setting though working earlier on windows xp. http_proxy = "proxy name:80" now this same setting doesn't work, i tried doing in the cmd.exe prompt. set http_proxy "proxy name:80" P.S. i am a normal user and don't have admin privleges. Regards, Om Prakash -- https://mail.python.org/mailman/listinfo/python-list
Re: ANN: binario - simple work with binary files
Rustom Mody wrote:
>On Tuesday, August 26, 2014 6:58:42 AM UTC+5:30, Tim Roberts wrote:
>> To the equivalent code with struct:
>
>> import struct
>
>> dscrp = "H?fs5B"
>
>> f = open('file.dat')
>> stuff = struct.unpack( dscrp, f.read() )
>
>> print stuff
>
>> In both cases, you have to KNOW the format of the data beforehand. If you
>> do a read_short where you happen to have written a float, disaster ensues.
>
>> I don't really see that you've added very much.
>
>I thought much the same.
>However notice your f.read(). Its type is string.
>
>What if file.dat is a 1GB wav file?
f.seek(512000)
stuff = struct.unpack( dscrp, f.read(128) )
The point is that files already know how to position themselves.
--
Tim Roberts, t...@probo.com
Providenza & Boekelheide, Inc.
--
https://mail.python.org/mailman/listinfo/python-list
Re:error while writing program to send mail.
Om Prakash Wrote in message:
> Hi,
>
> I am writing this program from
> https://docs.python.org/2/library/email-examples.html
>
> but getting the error as
>
> singhom@debian:~/pythons$ python send_email.py
> Traceback (most recent call last):
>File "send_email.py", line 18, in
> msg['Subject'] = 'The contents of $s' % message
> NameError: name 'message' is not defined
>
>
> i know the error would be something simple and i am overlooking it, any
> help would be highly appreciated, I am sorry, but I am very new to
> python programming.
>
> code starts here.
> #/usr/bin/python2.7 -tt
>
> ## sending a simple text message using python.
> import smtplib
>
> from email.mime.text import MIMEText
>
> # Open file for reading.
> fp = open("message", 'rb')
> # Create a plain text message.
> msg = MIMEText(fp.read())
>
> fp.close
This line doesn't do anything. If you want to close the file,
you need some parens, Or you can use the 'with' form instead of
open.
>
> me = "torque.in...@gmail.com"
> you = "oomprak...@gmail.com"
>
> msg['Subject'] = 'The contents of $s' % message
Just what did you plan to use here? You never defined a variable
called message. If you intended to use msg, I presume you realize
it could be quite large,
--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list
Re: error while writing program to send mail.
On 2014-09-02 00:35, Om Prakash wrote:
Hi,
I am writing this program from
https://docs.python.org/2/library/email-examples.html
but getting the error as
singhom@debian:~/pythons$ python send_email.py
Traceback (most recent call last):
File "send_email.py", line 18, in
msg['Subject'] = 'The contents of $s' % message
NameError: name 'message' is not defined
i know the error would be something simple and i am overlooking it, any
help would be highly appreciated, I am sorry, but I am very new to
python programming.
code starts here.
#/usr/bin/python2.7 -tt
## sending a simple text message using python.
import smtplib
from email.mime.text import MIMEText
# Open file for reading.
fp = open("message", 'rb')
# Create a plain text message.
msg = MIMEText(fp.read())
fp.close
That should be:
fp.close()
me = "torque.in...@gmail.com"
you = "oomprak...@gmail.com"
msg['Subject'] = 'The contents of $s' % message
You're trying to use the format operator, but:
1. You never bound the name 'message' to a value, hence:
NameError: name 'message' is not defined
2. The format string contains '$s' instead of '%s'.
msg['From'] = me
msg['To'] = you
# Send message thorugh localhost but don't include the envelope headers.
s = smtplib.SMTP('localhost')
s.sendmail(me, [you], msg.as_string())
s.quit()
--
https://mail.python.org/mailman/listinfo/python-list
Re: error while writing program to send mail.
On 02Sep2014 05:05, Om Prakash wrote: I am writing this program from https://docs.python.org/2/library/email-examples.html but getting the error as singhom@debian:~/pythons$ python send_email.py Traceback (most recent call last): File "send_email.py", line 18, in msg['Subject'] = 'The contents of $s' % message NameError: name 'message' is not defined i know the error would be something simple and i am overlooking it, any help would be highly appreciated, I am sorry, but I am very new to python programming. This problem is generic to almost any programming language: you have not defined the variable "message" before you try to use it in the format string. In the example code, the variable is called "textfile" and is supposed to be the name of a file containing some message text. The example code does not define it; you need to do so. After you sort that, you then have a ython problem: you have said "$s" in your format string, but the python "%" operator expected "%s" in the format string. Sort these two things and see how things proceed. Cheers, Cameron Simpson Personally, I find blinking text on a page to be extraordinarily annoying and my first instinct is to back up one page, right away. - William Barr -- https://mail.python.org/mailman/listinfo/python-list
error while writing program to send mail.
Hi,
I am writing this program from
https://docs.python.org/2/library/email-examples.html
but getting the error as
singhom@debian:~/pythons$ python send_email.py
Traceback (most recent call last):
File "send_email.py", line 18, in
msg['Subject'] = 'The contents of $s' % message
NameError: name 'message' is not defined
i know the error would be something simple and i am overlooking it, any
help would be highly appreciated, I am sorry, but I am very new to
python programming.
code starts here.
#/usr/bin/python2.7 -tt
## sending a simple text message using python.
import smtplib
from email.mime.text import MIMEText
# Open file for reading.
fp = open("message", 'rb')
# Create a plain text message.
msg = MIMEText(fp.read())
fp.close
me = "torque.in...@gmail.com"
you = "oomprak...@gmail.com"
msg['Subject'] = 'The contents of $s' % message
msg['From'] = me
msg['To'] = you
# Send message thorugh localhost but don't include the envelope headers.
s = smtplib.SMTP('localhost')
s.sendmail(me, [you], msg.as_string())
s.quit()
--
https://mail.python.org/mailman/listinfo/python-list
Re: Psycopg2 package installation puzzle in Pycharm - any thoughts?
Mark - it's more that I just didn't understand what you mean. Here's you: Probably an out and out programmer; uses a number of languages; a decade of experience, educated in best practice via your experience. Here's me: Idiot. A decade of experience in VBA and Excel in mindless finance jobs, hoping to build a 'not shit" website for a refrigerated van courier company I am starting. I know enough to find my way around linux and mac for most things and that Python and Django are a great tool, yet I stumble in my environment setup. I have built a Django website before using Pycharm on an Ubuntu machine without this problem. So I wasn't seeking to be abrasive, it's just that I don't get it, nor do I understand the boundary of "it". Here's what I can do: 1. I will lookup Thunderbird and see if this makes posting easier to follow 2. I will reverse my launchctl setenv changes. 3. Is there anything else I can do to improve diagnosis and communication? Andy -- https://mail.python.org/mailman/listinfo/python-list
Re: Editing text with an external editor in Python
On Tue, Sep 2, 2014 at 4:23 AM, Steven D'Aprano wrote: > Chris Angelico wrote: >> C:\>Python34\python 123123123.py >> cygwin warning: >> MS-DOS style path detected: C:\DOCUME~1\M\LOCALS~1\Temp\tmp94rcwd57 >> Preferred POSIX equivalent is: /DOCUME~1/M/LOCALS~1/Temp/tmp94rcwd57 > > That's arguably a Python bug. Under Cygwin, it should use POSIX paths rather > than Windows paths. Except that I wasn't; I ran Python 3.4 that was installed via the .msi package, and from that Python ran nano that was presumably compiled for Cygwin. >> Windows doesn't have a nice $EDITOR environment variable to call on, > > Why not? It's your environment, you can create any environment variable you > like, even under Windows, right? Sure, but what I mean is, there's a general convention on Unix that setting EDITOR will do that. You don't get to take advantage of expectations that easily on Windows. > But fundamentally, the de facto "standard editor" on Windows is Notepad. Sadly so. Which is why I tried it... >> You'll also have to cope with some other possibilities. What happens >> if someone tries Notepad? (Don't try this at home. We are experts and >> are testing on a closed track. Do not use Notepad unless you, too, >> have thirty years of special effects experience.) Turns out it doesn't >> like working with a file that another process has open. > > Ah, I feared that would be the case. I'll have to think about a way around > that. It won't be as neat, or as secure, but it should be doable. ... and yeah. That's the problem. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Editing text with an external editor in Python
On Tue, Sep 2, 2014 at 4:02 AM, Steven D'Aprano wrote: > I'm not really seeing how this is a security vulnerability. If somebody can > break into my system and set a hostile GIT_EDITOR, or TMPDIR, environment > variables, I've already lost. Agreed. If I'm calling on your program and setting EDITOR or GIT_EDITOR or whatever to configure how you ask me to edit a file, that's because it's *my* system. The aforementioned setup is actually run as root; the 'editor' quite deliberately does almost nothing, but I know it's safe because I'm the one in control, not because the editor's sanitized. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Editing text with an external editor in Python
On 02Sep2014 04:02, Steven D'Aprano wrote: Roy Smith wrote: Hmmm. Didn't we just have a thread about passing external data to shells? $ mkdir '/tmp/;rm -rf;' $ TMPDIR='/tmp/;rm -rf;' python Python 2.7.3 (default, Sep 26 2013, 20:03:06) [GCC 4.6.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. import tempfile f = tempfile.NamedTemporaryFile() f.name '/tmp/;rm -rf;/tmpW8HFTr' Seems like a lot of trouble to go to to erase your own system. Couldn't you just run rm -rf / on your own system prior to launching Python? But seriously, I'm not sure what attack vector you think you have found. By definition, this is calling out to an external application, which might do *anything*. It needs to be used in a trusted environment, like any other tool which calls out to external applications. [...] I'm not really seeing how this is a security vulnerability. If somebody can break into my system and set a hostile GIT_EDITOR, or TMPDIR, environment variables, I've already lost. [...] Have I missed something? I really don't think this is a vulnerability, and I don't see how using the subprocess module would make it safer. It is not just about being hacked. It is about being robust in the face of unusual setups. If I were producing this function for general use (even my own personal general use) it would need to be reliable. That includes things like $TMPDIR having spaces in it (or other unfortunate punctuation). On any system where people use GUIs to manipulate files and folders, having spaces and arbitrary punctuation in pathnames is common. Pointing $TMPDIR at such a place for a special purpose is not unreasonable. People keep assuming injection is all about malice and being hacked. It is not. It is also about robustness and reliability, and possible silent failure/misfunction. Cheers, Cameron Simpson st...@ensoniq.com says... | Motorcycle maintenence is an art, isn't it? By the time you've finished, it's a black art. - Dave Parry -- https://mail.python.org/mailman/listinfo/python-list
Re: Information
On Mon, Sep 1, 2014 at 3:40 PM, wrote: > Hey > I am Getachew , I am using cantera, python xy 2.7.6. My question is how can i > convert XML file to Ct. py file format. > > > Thanks > -- > https://mail.python.org/mailman/listinfo/python-list This is a general purpose python list. Googling a bit it looks like you are doing something in the genetics field. Python has several xml parsers -- Joel Goldstick http://joelgoldstick.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Editing text with an external editor in Python
On 2014-09-02 04:23, Steven D'Aprano wrote: > Read $VISUAL, if it exists, otherwise $EDITOR, if it exists, > otherwise fall back on something hard coded. Or read it from an ini > file. Or create an entry in the register. Whatever. That's up to > the application which uses this function, not the function itself. > > Under XP and older the standard DOS editor is EDIT, but that's gone > from Windows 7. Believe it or not, I understand that you can use: > > copy con [filename.???] > > to do basic line editing, which is terrifying, but I believe it > works. And according to [1], the venerable edlin is still available on Win8, even if you don't have edit. Though according to [2], it sounds like MS-EDIT is still available on at least the 32-bit version of Win8 (it doesn't detail whether it comes out of the box on 64-bit Win8). I don't have Win8, so I can't corroborate either application. -tkc [1] http://en.wikipedia.org/wiki/Edlin#History [2] http://en.wikipedia.org/wiki/MS-DOS_Editor -- https://mail.python.org/mailman/listinfo/python-list
Re: subprocess module usage
Earl Lapus writes: > Hi, > > I made simple test program using the subprocess module (see attached: > exec_cmd.py). I ran it passing variations of 'ls' command options. > > I encounter exceptions every time I use '-l' options. Example runs > where exception occurs: > # ./exec_cmd.py ls -al > # ./exec_cmd.py ls -l > > However, if I pass 'ls' and arguments as one argument, like so: > #./exec_cmd.py 'ls -al' > exception does not occur. > > I logged output (see ls_test.out). > > So, what could be causing this behavior? Is this expected or is there > something wrong with how I'm using the subprocess module? You shouldn't use shell=True with a list argument (sys.argv[1:] is a list). Specify the command as a string or use shell=False. See http://bugs.python.org/issue21347 -- Akira -- https://mail.python.org/mailman/listinfo/python-list
Information
Hey I am Getachew , I am using cantera, python xy 2.7.6. My question is how can i convert XML file to Ct. py file format. Thanks -- https://mail.python.org/mailman/listinfo/python-list
Re: Editing text with an external editor in Python
Chris Angelico wrote: > On Tue, Sep 2, 2014 at 2:11 AM, Steven D'Aprano > wrote: >> Anyone able to test it on Windows for me please? >> > > Seems to partially work. I added an 'import os' at the top, and a > simple test call to the function, and it did give me my editor (nano) > and retrieved the text. It did give a warning, though: > > > C:\>Python34\python 123123123.py > cygwin warning: > MS-DOS style path detected: C:\DOCUME~1\M\LOCALS~1\Temp\tmp94rcwd57 > Preferred POSIX equivalent is: /DOCUME~1/M/LOCALS~1/Temp/tmp94rcwd57 That's arguably a Python bug. Under Cygwin, it should use POSIX paths rather than Windows paths. I believe that sys.platform tells you if you are running under Cygwin. > Windows doesn't have a nice $EDITOR environment variable to call on, Why not? It's your environment, you can create any environment variable you like, even under Windows, right? > so I'm not sure what the best way to actually choose an editor is. I > would hope that you can make it externally configurable Read $VISUAL, if it exists, otherwise $EDITOR, if it exists, otherwise fall back on something hard coded. Or read it from an ini file. Or create an entry in the register. Whatever. That's up to the application which uses this function, not the function itself. Under XP and older the standard DOS editor is EDIT, but that's gone from Windows 7. Believe it or not, I understand that you can use: copy con [filename.???] to do basic line editing, which is terrifying, but I believe it works. http://stackoverflow.com/a/22000756 And of course, you can install whatever third-party editors you like. There are Windows ports of nano, vim and emacs. But fundamentally, the de facto "standard editor" on Windows is Notepad. > You'll also have to cope with some other possibilities. What happens > if someone tries Notepad? (Don't try this at home. We are experts and > are testing on a closed track. Do not use Notepad unless you, too, > have thirty years of special effects experience.) Turns out it doesn't > like working with a file that another process has open. Ah, I feared that would be the case. I'll have to think about a way around that. It won't be as neat, or as secure, but it should be doable. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Editing text with an external editor in Python
Roy Smith wrote:
> Hmmm. Didn't we just have a thread about passing external data to
> shells?
>
> $ mkdir '/tmp/;rm -rf;'
> $ TMPDIR='/tmp/;rm -rf;' python
> Python 2.7.3 (default, Sep 26 2013, 20:03:06)
> [GCC 4.6.3] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
import tempfile
f = tempfile.NamedTemporaryFile()
f.name
> '/tmp/;rm -rf;/tmpW8HFTr'
Seems like a lot of trouble to go to to erase your own system. Couldn't you
just run rm -rf / on your own system prior to launching Python?
But seriously, I'm not sure what attack vector you think you have found.
By definition, this is calling out to an external application, which might
do *anything*. It needs to be used in a trusted environment, like any other
tool which calls out to external applications.
[steve@ando ~]$ mkdir foo
[steve@ando ~]$ cd foo
[steve@ando foo]$ git init
Initialized empty Git repository in /home/steve/foo/.git/
[steve@ando foo]$ echo Some content > stuff.txt
[steve@ando foo]$ git add stuff.txt
[steve@ando foo]$ GIT_EDITOR="echo 'you got pwned' #" git commit
you got pwned
Aborting commit due to empty commit message.
I'm not really seeing how this is a security vulnerability. If somebody can
break into my system and set a hostile GIT_EDITOR, or TMPDIR, environment
variables, I've already lost.
As written, the edit() function takes two arguments: the name of an
application to call, and optionally initial contents to be edited.
Obviously the name of the application has to be trusted: either hard code
it, or get it from a trusted source like the VISUAL or EDITOR environment
variables. (It's *your* environment, you can set them to whatever editor
you want. If you want to set them to something hostile, that's your
prerogative.)
If an attacker has already compromised my editor, I've lost.
If I naively run arbitrary code provided by *untrusted* sources (say, I get
the editor from anonymous users over the internet), I've lost. I didn't
think I needed to explicitly say that.
On the other hand, the initial content need not be trusted, since it's just
text. The worst somebody could do is hurt my feelings. (Well, they could in
principle buffer-overflow the editor, or Python, but if you can't trust
Python and your editor to be resistant to that, you shouldn't use them.) If
the initial content could "leak" out and do bad things, that would be a
vulnerability I care about. Having the user destroy their own system by
deliberately misusing the function is not my problem:
# Don't do this. It would be bad.
edit("rm -rf / #")
Have I missed something? I really don't think this is a vulnerability, and I
don't see how using the subprocess module would make it safer.
--
Steven
--
https://mail.python.org/mailman/listinfo/python-list
Re: Editing text with an external editor in Python
In article <54049ab7$0$29972$c3e8da3$54964...@news.astraweb.com>, Steven D'Aprano wrote: > import tempfile > > def edit(editor, content=''): > f = tempfile.NamedTemporaryFile(mode='w+') > [...] > command = editor + " " + f.name > status = os.system(command) Hmmm. Didn't we just have a thread about passing external data to shells? $ mkdir '/tmp/;rm -rf;' $ TMPDIR='/tmp/;rm -rf;' python Python 2.7.3 (default, Sep 26 2013, 20:03:06) [GCC 4.6.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tempfile >>> f = tempfile.NamedTemporaryFile() >>> f.name '/tmp/;rm -rf;/tmpW8HFTr' >>> -- https://mail.python.org/mailman/listinfo/python-list
Re: Editing text with an external editor in Python
On Tue, Sep 2, 2014 at 2:11 AM, Steven D'Aprano wrote: > Anyone able to test it on Windows for me please? > Seems to partially work. I added an 'import os' at the top, and a simple test call to the function, and it did give me my editor (nano) and retrieved the text. It did give a warning, though: C:\>Python34\python 123123123.py cygwin warning: MS-DOS style path detected: C:\DOCUME~1\M\LOCALS~1\Temp\tmp94rcwd57 Preferred POSIX equivalent is: /DOCUME~1/M/LOCALS~1/Temp/tmp94rcwd57 CYGWIN environment variable option "nodosfilewarning" turns off this warning. Consult the user's guide for more details about POSIX paths: http://cygwin.com/cygwin-ug-net/using.html#using-pathnames Your text is: asdf Hello, world! Windows doesn't have a nice $EDITOR environment variable to call on, so I'm not sure what the best way to actually choose an editor is. I would hope that you can make it externally configurable (I've done some very weird things with changed editors, like one that connects to a TCP socket, alerts a server with its arguments, and then SIGSTOPs itself, and the server sends the file's contents to another socket-connected client that has a human at the other end, and when it gets back a response from that client, it rewrites the file and SIGCONTs the 'editor', which then terminates - so to all intents and purposes, it's as if that program really did edit the file), but doing that on Windows may not be easy. You'll also have to cope with some other possibilities. What happens if someone tries Notepad? (Don't try this at home. We are experts and are testing on a closed track. Do not use Notepad unless you, too, have thirty years of special effects experience.) Turns out it doesn't like working with a file that another process has open. Nor can SciTE; it shows an empty file on load, and then is unable to save. I suspect the comment here is what's biting you: https://docs.python.org/3.5/library/tempfile.html#tempfile.NamedTemporaryFile """Whether the name can be used to open the file a second time, while the named temporary file is still open, varies across platforms (it can be so used on Unix; it cannot on Windows NT or later).""" So it works fairly nicely as long as you're using a Cygwin editor. Otherwise, not so much. :( ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Editing text with an external editor in Python
Python's input() or raw_input() function is good for getting a single line of text from the user. But what if you want a more substantial chunk of text from the user? Here's how to call out to an external editor such as ed, nano, vim, emacs, and even GUI text editors: import tempfile def edit(editor, content=''): f = tempfile.NamedTemporaryFile(mode='w+') if content: f.write(content) f.flush() command = editor + " " + f.name status = os.system(command) f.seek(0, 0) text = f.read() f.close() assert not os.path.exists(f.name) return (status, text) Anyone able to test it on Windows for me please? More here: https://code.activestate.com/recipes/578926/ -- Steven -- https://mail.python.org/mailman/listinfo/python-list
[Announce] tabhistory - tab completion and command history
I am happy to announce an upgrade to the tabhistory module, which brings advanced tab completion and command history to Python 2.4 through 3.3 and beyond. Features Tab completion -- * At the beginning of lines, pressing the TAB key indents the line. * Inside strings, pressing TAB performs filename completion of the string. * Following the "import" and "from" keywords, pressing TAB completes on modules and module attributes. For security, module attributes are only checked if the module has already been imported and is in the sys.modules cache. * Everywhere else, pressing TAB completes on keywords, built-ins, globals, and references of the form "name.name". * Supports readline key binding and editing commands. Command history --- * Save command line history during interactive sessions, and automatically restore that history when you start up again. * Use the up-arrow to call up previous commands, and down-arrow for more recent ones. * Display the last N command lines. === Where to get it === Clone the Mercurial repository: hg clone https://code.google.com/p/tabhistory/ There is a single Python file, tabhistory.py. Put it somewhere on your Python path, launch the interactive interpreter, and run: import tabhistory and it will be enabled. If you don't have Mercurial, you can download the source code from here: http://code.google.com/p/tabhistory/ === Limitations === * This may not work on Windows at all. In theory, it might work if you install the pyreadline third-party module, but I have no way of testing this. Feedback from Windows users will be gratefully accepted. * There may be some issues on Mac OS X or other Unixes which use libedit instead of libreadline. I will be grateful for comments and bug fixes from any libedit users. * Python 3.0 is not officially supported (but it probably will work). Bug reports, feature requests and other comments are welcome. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Psycopg2 package installation puzzle in Pycharm - any thoughts?
On 01/09/2014 13:32, andydtay...@gmail.com wrote: Posting style point taken. Google groups doesn't exactly help you with that. Thunderbird is as good a solution as any although there are plenty of other choices. * Statements like "Please equip yourself with a tool that provides us with some context" add nothing and are not exactly community inclusive. So you promptly send another message and still no context, what do you think we are, mind readers? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: Psycopg2 package installation puzzle in Pycharm - any thoughts?
On Mon, Sep 1, 2014 at 10:32 PM, wrote: > Posting style point taken. Google groups doesn't exactly help you with that. > > * You guys have probably been tinkering with this stuff for years. I haven't. > * Your man on the street would say I described the error fairly well. > * It's not like I wasn't trying my best to fix it myself > * So far as environment tinkering is concerned I have entered a "launchctl > setenv PATH" statement in two places. Both reversible. > * I think what I have is an issue pertaining to OSX Mavericks and Pycharm. If > you have nothing to add, just say it. > * Statements like "Please equip yourself with a tool that provides us with > some context" add nothing and are not exactly community inclusive. Part of the point is that Google Groups is quite unhelpful in these areas. There are alternatives - you can get a newsreader, or you can join the mailing list. (I do the latter, and Gmail works fairly well.) Sometimes, the community is best served by telling people about flawed tools. It's more courteous than saying "You didn't quote any text, and that's making it really hard for us to follow" and blaming you personally for it :) But the fact is that it *does* make things harder for other people, and the onus is on you to follow the conventions of courtesy. That's why people are asking you to change software - because good software makes it easy to be courteous. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Psycopg2 package installation puzzle in Pycharm - any thoughts?
On 2014-09-01 12:32:38 +, andydtay...@gmail.com said: Google groups doesn't exactly help you with that. Drop it, get a usenet client or subscribe the mailing list (the newsgroup and the ml are bridged IIRC). * Your man on the street would say I described the error fairly well. That man probably doesn't try to solve other people's issues. * So far as environment tinkering is concerned I have entered a "launchctl setenv PATH" statement in two places. Both reversible. That will affect your session environment (that is separated from your terminal environment), but since PyCharm doesn't pick a Python interpreter from PATH but rather let the user explicitly select one you're more intersted in just selecting a correct interpreter. * I think what I have is an issue pertaining to OSX Mavericks and Pycharm. If you have nothing to add, just say it. I forgot to mention it but I'm on Mavericks and downloaded the latest PyCharm just to check you issue, I never used it and just followed the new project wizard that asked me to select a python interpeter or to create a virtualenv. * Statements like "Please equip yourself with a tool that provides us with some context" add nothing and are not exactly community inclusive. Still you took the time to write the message I'm replying to, that won't help you much, rather than running """ import sys print(sys.executable) """ in your project, that would provide more information about your issue. -- Andrea -- https://mail.python.org/mailman/listinfo/python-list
Re: Psycopg2 package installation puzzle in Pycharm - any thoughts?
Posting style point taken. Google groups doesn't exactly help you with that. * You guys have probably been tinkering with this stuff for years. I haven't. * Your man on the street would say I described the error fairly well. * It's not like I wasn't trying my best to fix it myself * So far as environment tinkering is concerned I have entered a "launchctl setenv PATH" statement in two places. Both reversible. * I think what I have is an issue pertaining to OSX Mavericks and Pycharm. If you have nothing to add, just say it. * Statements like "Please equip yourself with a tool that provides us with some context" add nothing and are not exactly community inclusive. -- https://mail.python.org/mailman/listinfo/python-list
Re: subprocess module usage
On Mon, Sep 1, 2014 at 6:46 PM, Cameron Simpson wrote: > Not really. If the arguments are coming in from the command line, someone (a > user, even if that user is the programmer) typed them. Even if not > malicious, they can still be mistaken. Or just unfortunate. I'm guessing that what he means is that the example posted here used sys.argv but his actual code doesn't. It's still important to *understand* shell=True, but it can be perfectly safe to use it. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: subprocess module usage
On 01Sep2014 14:33, Earl Lapus wrote: On Mon, Sep 1, 2014 at 1:39 PM, Chris Angelico wrote: Glad it's working! But please, don't just take my word for it and make a black-box change to your code. When you invoke subprocesses, be sure you understand what's going on, and when shell=True is appropriate and when shell=False is appropriate. The docs should be fairly clear on this. If you get this sort of thing wrong, you'll get weird errors like this (if you're lucky), or open yourself up to shell injection vulnerabilities (if you're not). The command and arguments that will be passed to check_output will not depend on user input. So, the chances of malicious commands from being executed would be low (right?). Not really. If the arguments are coming in from the command line, someone (a user, even if that user is the programmer) typed them. Even if not malicious, they can still be mistaken. Or just unfortunate. You should always want to do exactly what you're asked. If you misuse shell=True when the user is expecting shell=False (i.e. "just do what I said!"), then your program will not carry out the user's intent. If it does not fail outright, it will presumably do the _wrong_ thing. Cheers, Cameron Simpson Music journalism: People who can't write interviewing people who can't talk for people who can't read. - Frank Zappa -- https://mail.python.org/mailman/listinfo/python-list
Re: This could be an interesting error
Steven D'Aprano wrote: or words in Foreign like "cwm" Seeing that "w" is a vowel in Welsh, there should probably be a special version of the program for Welsh speakers. (Welshlatin? Pigwelsh?) -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: Psycopg2 package installation puzzle in Pycharm - any thoughts?
You make hard to follow your messages both by sending lot of messages and not using an adequate quoting. Please pick a posting style [1] (possibly interleaved) and stick to it. On 2014-08-31 23:35:09 +, andydtay...@gmail.com said: Andrea - yes I am using the virtualenv interpreter as the Pycharm project interpreter That statement needs to be backed by some proof, saying "I'm using such interpreter" doesn't necessarily mean it's true, print out sys.executable and check. I confirm I just downloaded PyCharm CE 3.4.1 (you didn't provide your PyCharm version), set up a virtualenv, created an empty project using the interpreter from that environment and was able to run the file with the import statement in PyCharm, so the IDE works as expected. It was actually quite straightforward and PyCharm also offered to create the virtualenv while selecting the interpreter. I suggest to stop messing with your environment (I would revert all those changes) and figure what the problem actually is. [1] http://en.wikipedia.org/wiki/Posting_style -- Andrea -- https://mail.python.org/mailman/listinfo/python-list
