SQLObject 0.13.1
Hello! I'm pleased to announce version 0.13.1, a minor bugfix release of branch 0.13 of SQLObject. What is SQLObject = SQLObject is an object-relational mapper. Your database tables are described as classes, and rows are instances of those classes. SQLObject is meant to be easy to use and quick to get started with. SQLObject supports a number of backends: MySQL, PostgreSQL, SQLite, Firebird, Sybase, MSSQL and MaxDB (also known as SAPDB). Where is SQLObject == Site: http://sqlobject.org Development: http://sqlobject.org/devel/ Mailing list: https://lists.sourceforge.net/mailman/listinfo/sqlobject-discuss Archives: http://news.gmane.org/gmane.comp.python.sqlobject Download: http://pypi.python.org/pypi/SQLObject/0.13.1 News and changes: http://sqlobject.org/News.html What's New == News since 0.13.0 - * A bug was fixed in a subtle case when a per-instance connection is not passed to validators. For a more complete list, please see the news: http://sqlobject.org/News.html Oleg. -- Oleg Broytmanhttp://phd.pp.ru/p...@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
SQLObject 0.14.1
Hello! I'm pleased to announce version 0.14.1, a minor bugfix release of branch 0.14 of SQLObject. What is SQLObject = SQLObject is an object-relational mapper. Your database tables are described as classes, and rows are instances of those classes. SQLObject is meant to be easy to use and quick to get started with. SQLObject supports a number of backends: MySQL, PostgreSQL, SQLite, Firebird, Sybase, MSSQL and MaxDB (also known as SAPDB). Where is SQLObject == Site: http://sqlobject.org Development: http://sqlobject.org/devel/ Mailing list: https://lists.sourceforge.net/mailman/listinfo/sqlobject-discuss Archives: http://news.gmane.org/gmane.comp.python.sqlobject Download: http://pypi.python.org/pypi/SQLObject/0.14.1 News and changes: http://sqlobject.org/News.html What's New == News since 0.14.0 - * A bug was fixed in a subtle case when a per-instance connection is not passed to validators. For a more complete list, please see the news: http://sqlobject.org/News.html Oleg. -- Oleg Broytmanhttp://phd.pp.ru/p...@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
ANNOUNCE: Gaphor 0.16.0 / Gaphas 0.7.0
Gaphor 0.16.0 = Gaphor is a UML modeling tool written in Python. The code is stable for a while now, so it's about time we make a new release. Since the last version some nice new functionality has been added. Most notably: - Guides support from Gaphas 0.7.0 - hand written drawing style - Keyboard shortcuts for toolbox selection - Fixed issue in undo transaction handling - Proper dialog when exiting with changed model --- Gaphas (the canvas component) has been released a while ago. The most important changes are: - New feature: Guides, for aligning elements - Free hand drawing style - Introduced aspects for finding items and handles - Painters are bound to a specific view, like tools It's a worthwhile update! Regards, Arjan -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
Re: Boolean value of generators
Paul Rubin no.em...@nospam.invalid writes: Steven D'Aprano st...@remove-this-cybersource.com.au writes: (4) Expensive generators. The beauty of generators is that they produce values on demand. Making all generators cache their first value means that you pay that cost even if you end up never needing the first value. You wouldn't generate the cached value ahead of time. You'd just remember the last generated value so that you could use it again. Sort of like getc/ungetc. An intermediate measure might be to have a stdlib wrapper that added caching like this to an arbitrary generator. I've written such things a few times in various clumsy ways. Having the caching available in the C code would eliminate a bunch of indirection. I've done such a thing myself a few times. I remember posting on python-ideas a while ago (no time to find the thread ATM). My suggestion was to add a function peekable(it) that returns an iterator with a peek() method, whose behaviour is exactly the one that you describe (i.e. similar to getc/ungetc). I also suggested that iterators could optionally implement a peek() method themselves, in which case peek(it) would return the iterator without modification. For examples, list_iterators, str_iterators and other iterators over sequences could implement next() without any cost. I don't recall that this proposal gained much traction! -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: what happens to Popen()'s parent-side file descriptors?
On Thu, 14 Oct 2010 08:48:45 -0700, Roger Davis wrote: On a related point here, I have one case where I need to replace the shell construct externalprog somefile otherfile I suppose I could just use os.system() here but I'd rather keep the Unix shell completely out of the picture (which is why I am moving things to Python to begin with!), so I'm just doing a simple open() on somefile and otherfile and then passing those file handles into Popen() for stdin and stdout. I am already closing those open()ed file handles after the child completes, but I suppose that I probably should also explicitly close Popen's p.stdin and p.stdout, too. (I'm guessing they might be dup()ed from the original file handles?) p.stdin will be None unless you use stdin=subprocess.PIPE; similarly for stdout. Another gotcha regarding pipes: the reader only sees EOF once there are no writers, i.e. when the *last* writer closes their end. If Python has a descriptor for the write end of a pipe, any child process will inherit it unless you use close_fds=True or close it via a function specified by the preexec_fn argument. Allowing it to be inherited can prevent the reader from seing EOF on the pipe. E.g. if you do: p1 = Popen(..., stdin = PIPE, stdout = PIPE) p2 = Popen(..., stdin = p1.stdout) p2 will inherit p1.stdin (that's the write end) from Python. Subsequently calling p1.stdin.close() *won't* cause p1 to see EOF on its stdin because p2 still has its inherited copy of the descriptor open. On Windows, only stdin, stdout and stderr are inherited, so this isn't an issue there. -- http://mail.python.org/mailman/listinfo/python-list
Re: what happens to Popen()'s parent-side file descriptors?
In message i968f501...@news6.newsguy.com, Chris Torek wrote: Running the above code fragment in a different implementation, in which garbage collection is deferred, would *not* close the file descriptor, and the system would potentially run out (depending on when a gc occurred, and/or whether the system would attempt gc on running out of file descriptors, in the hope that the gc would free some up). Aren’t you glad IronPython is dead? -- http://mail.python.org/mailman/listinfo/python-list
Re: what happens to Popen()'s parent-side file descriptors?
In message 8bec27dd-b1da-4aa3-81e8-9665db040...@n40g2000vbb.googlegroups.com, Roger Davis wrote: Documentation on python.org states that GC can be postponed or omitted altogether ... Yes, but no sensible Python implementation would do that, as it’s a recipe for resource bloat. -- http://mail.python.org/mailman/listinfo/python-list
Re: Weird try-except vs if behavior
On Thu, 2010-10-14 at 22:43 -0700, James Matthews wrote: Hi, I have this code http://gist.github.com/627687 (I don't like pasting code into the mailing list). Yes, but it makes it harder to discuss the ode and makes the archive that much more useless. It's only a couple of lines, here ya go: def tryway(): try: while True: alist.pop() except IndexError: pass def ifway(): while True: if blist == []: break else: blist.pop() if __name__=='__main__': alist = range(1000) blist = range(1000) from timeit import Timer print Testing Try tr = Timer(tryway(),from __main__ import tryway) print tr.timeit() print Testing If ir = Timer(ifway(),from __main__ import ifway) print ir.timeit() I am wondering why the try except is taking longer. I assume that if the IF statement checks every iteration of the loop (1000 times) shouldn't it be slower? You're not measuring what you think. The way you've structured your code, alist and blist are module globals. Timeit will call the tryway function one million times in a loop. The first time it works as you expect, popping each element off of alist in turn. On the other 99 executions, alist is already empty (having been emptied by the first execution) and the IndexError is raised immediately. Likewise for the execution of ifway. So what you're really measuring is the overhead of catching an exception versus testing a condition for an *empty* list - no surprise that the conditional version is faster! If you re-initialise the list at the beginning of each function, the try-based version is faster as you expect. I.e.: def tryway(): alist = range(1000) try: while True: alist.pop() except IndexError: pass I get the following times for a thousand iterations (not waiting around for a million!) Testing Try 0.224129915237 Testing If 0.300312995911 Cheers, Ryan -- Ryan Kelly http://www.rfk.id.au | This message is digitally signed. Please visit r...@rfk.id.au| http://www.rfk.id.au/ramblings/gpg/ for details signature.asc Description: This is a digitally signed message part -- http://mail.python.org/mailman/listinfo/python-list
Re: PyCharm
On 2010-10-13 23:36:31 +0200, Robert H said: Since the new IDE from Jetbrains is out I was wondering if you are using it and what you think about it. It sucks. http://regebro.wordpress.com/2010/10/14/python-ide-code-completion-test Kai -- http://mail.python.org/mailman/listinfo/python-list
Re: optparser: how to register callback to display binary's help
On Oct 13, 6:03 pm, Jean-Michel Pichavant jeanmic...@sequans.com
wrote:
hiral wrote:
Hi,
I want to display help message of python script and then display help
message from the binary file (which also supports -h option):
Assumptions:
1) 'mybinary' - is linux executable file which supports '-h' and on '-
h' option it displays the help message
2) myscript.py - when passing '-h' option want to call 'mybinary -h'
and display help messages of both script and binary.
Code:
from optparse import OptionParser
[...]
parser = OptionParser()
parser.add_option(-e, --execute, dest=file,
help=Execute binary, metavar=FILE)
(options, args) = parser.parse_args()
if options.file:
subprocess.call(options.file)
Requirement:
$ python myscript.py -h
currently it prints the help message with '-e' and '-h' help message
becides I want to display help message of 'mybinary'
Thank you in advance.
-Hiral
Hi,
Try something like
def myHelp(option, opt, value, parser):
parser.print_help()
if hasattr(parser.values, 'file') and parser.values.file:
proc = subprocess.Popen([parser.values.file, '-h'],
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
print proc.stdout.read()
print proc.stderr.read()
parser.remove_option('-h')
parser.add_option(-h, --help,
help=display the help message, action='callback',
callback = myHelp)
Jean-Michel- Hide quoted text -
- Show quoted text -
Hi Jean-Michel,
It helped !!!
Thank you.
-Hiral
--
http://mail.python.org/mailman/listinfo/python-list
Is there any module for automated testing in python?
Hi, Is there any module for automated testing in python? Pls help me frns.. From Gopi -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there any module for automated testing in python?
On Fri, Oct 15, 2010 at 8:19 PM, gopi krishna dasarathulag...@gmail.com wrote: Hi, Is there any module for automated testing in python? Pls help me frns.. py.test is quite nice and I use it for my own project(s). The company I work for also uses it to test their software. cheers James -- -- James Mills -- -- Problems are solved by method -- http://mail.python.org/mailman/listinfo/python-list
Re: processing input from multiple files
Thanks,
The issue with the times is now sorted, however I'm running into a problem
towards the end of the script:
File sortoutsynop2.py, line 131, in module
newline =
message_type+c+str(station_id)+c+newtime+c+lat+c+lon+c+c+-+c+ 002
+c+-+c+-+c+str(pressure)+c
TypeError: cannot concatenate 'str' and 'list' objects
I think I can see the issue here, but I'm not entirely sure how to get
around it. Several of my variables change either from one file to the next
or from each line. Time and pressure would be examples of both of these
types.Yet others, such as message_type, are constant. As a result I have a
mixture of both lists and strings. Should I then create a list of the
constant values? I'm a little confused, I'll send you the script that works
for a single file and I'll see if I can come up with a more logical way
around it.
#!/usr/bin/python
import sys
import os
import re
#foutname = 'test.txt'
#filelist = os.system('ls
fname = datalist_201081813.txt
foutname1 = 'prestest.txt'
foutname2 = 'temptest.txt'
foutname3 = 'tempdtest.txt'
foutname4 = 'wspeedtest.txt'
foutname5 = 'winddtest.txt'
time = fname.split('_')[1].split('.')[0]
year = time[:4]
month = time[4:6]
day = time[6:8]
hour = time[-2:]
newtime = year+month+day+'_'+hour+''
c = ','
file1 = open(fname,r)
file2 = open(uk_stations.txt,r)
stations = file2.readlines()
ids=[]
names=[]
lats=[]
lons=[]
for item in stations:
item_list = item.strip().split(',')
ids.append(item_list[0])
names.append(item_list[1])
lats.append(item_list[2])
lons.append(item_list[3])
st = file1.readlines()
print st
data=[item[:item.find(' 333 ')] for item in st]
#data=st[split:]
print data
pres_out = ''
temp_out = ''
dtemp_out = ''
dir_out = ''
speed_out = ''
for line in data:
elements=line.split(' ')
station_id = elements[0]
try:
index = ids.index(station_id)
lat = lats[index]
lon = lons[index]
message_type = 'blah'
except:
print 'Station ID',station_id,'not in list!'
lat = lon = 'NaN'
message_type = 'Bad_station_id'
try:
temp = [item for item in elements if item.startswith('1')][0]
temperature = float(temp[2:])/10
sign = temp[1]
if sign == 1:
temperature=-temperature
except:
temperature='NaN'
try:
dtemp = [item for item in elements if item.startswith('2')][0]
dtemperature = float(dtemp[2:])/10
sign = dtemp[1]
if sign == 1:
dtemperature=-dtemperature
except:
detemperature='NaN'
try:
press = [item for item in elements[2:] if item.startswith('4')][0]
if press[1]=='9':
pressure = float(press[1:])/10
else:
pressure = float(press[1:])/10+1000
except:
pressure = 'NaN'
try:
wind = elements[elements.index(temp)-1]
direction = float(wind[1:3])*10
speed = float(wind[3:])*0.51444
except:
direction=speed='NaN'
newline =
message_type+c+str(station_id)+c+newtime+c+lat+c+lon+c+'-'+c+'002'+c+'-'+c+'-'+c+str(pressure)+c
print newline
pres_out+=newline+'\n'
newline2 =
message_type+c+str(station_id)+c+newtime+c+lat+c+lon+c+c+-+c+ 011
+c+-+c+-+c+str(temperature)+c
print newline2
temp_out+=newline2+'\n'
fout = open(foutname2,'w')
fout.writelines(temp_out)
fout.close()
newline3 =
message_type+c+str(station_id)+c+newtime+c+lat+c+lon+c+c+-+c+ 017
+c+-+c+-+c+str(dtemperature)+c
print newline3
dtemp_out+=newline3+'\n'
fout = open(foutname3,'w')
fout.writelines(dtemp_out)
fout.close()
newline4 =
message_type+c+str(station_id)+c+newtime+c+lat+c+lon+c+c+-+c+ 031
+c+-+c+-+c+str(direction)+c
print newline4
dir_out+=newline4+'\n'
fout = open(foutname4,'w')
fout.writelines(dir_out)
fout.close()
newline5 =
message_type+c+str(station_id)+c+newtime+c+lat+c+lon+c+c+-+c+
032+c+-+c+-+c+str(speed)+c
print newline5
speed_out+=newline5+'\n'
fout = open(foutname1,'w')
fout.writelines(pres_out)
fout.close()
fout = open(foutname2,'w')
fout.writelines(temp_out)
fout.close()
fout = open(foutname3,'w')
fout.writelines(dtemp_out)
fout.close()
fout = open(foutname4,'w')
fout.writelines(dir_out)
fout.close()
fout = open(foutname5,'w')
fout.writelines(speed_out)
fout.close()
cheers
Chris
On Thu, Oct 14, 2010 at 8:15 PM, John Posner jjpos...@optimum.net wrote:
On 10/14/2010 10:44 AM, Christopher Steele wrote:
The issue is that I need to be able to both, split the names of the files
so that I can extract the relevant times, and open each individual file and
process each line individually. Once I have achieved this I need to append
the sorted files onto one another in one long file so that I can pass them
into a verification package. I've tried changing the name to textline and I
get the
Re: Is there any module for automated testing in python?
gopi krishna, 15.10.2010 12:19: Is there any module for automated testing in python? What kind of automated testing do you mean? Generating test cases? Generating test runs from test data? Running unit tests automatically as part of a build? Running tests repeatedly to report changes in test results? There are several tools for Python that support running unit tests in one way or another. If you meant continuous integration, take a look at the Hudson CI server. Stefan -- http://mail.python.org/mailman/listinfo/python-list
How to display unicode char in Windows
Hi, I tried... code # coding: latin-1 print ** oo = ö print char=%s % oo print ** /code but it is not printing ö char; any idea? Thank you. -Hiral -- http://mail.python.org/mailman/listinfo/python-list
Re: How to display unicode char in Windows
Hi!
1) the good syntax is:
# -*- coding: latin-1 -*-
print **
oo = ö
print char=%s % oo
print **
2) in the console (commandLine), use this command: CHCP 1252 {Enter}
(before run your script)
@-salutations
--
Michel Claveau
--
http://mail.python.org/mailman/listinfo/python-list
SQLObject 0.13.1
Hello! I'm pleased to announce version 0.13.1, a minor bugfix release of branch 0.13 of SQLObject. What is SQLObject = SQLObject is an object-relational mapper. Your database tables are described as classes, and rows are instances of those classes. SQLObject is meant to be easy to use and quick to get started with. SQLObject supports a number of backends: MySQL, PostgreSQL, SQLite, Firebird, Sybase, MSSQL and MaxDB (also known as SAPDB). Where is SQLObject == Site: http://sqlobject.org Development: http://sqlobject.org/devel/ Mailing list: https://lists.sourceforge.net/mailman/listinfo/sqlobject-discuss Archives: http://news.gmane.org/gmane.comp.python.sqlobject Download: http://pypi.python.org/pypi/SQLObject/0.13.1 News and changes: http://sqlobject.org/News.html What's New == News since 0.13.0 - * A bug was fixed in a subtle case when a per-instance connection is not passed to validators. For a more complete list, please see the news: http://sqlobject.org/News.html Oleg. -- Oleg Broytmanhttp://phd.pp.ru/p...@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. -- http://mail.python.org/mailman/listinfo/python-list
SQLObject 0.14.1
Hello! I'm pleased to announce version 0.14.1, a minor bugfix release of branch 0.14 of SQLObject. What is SQLObject = SQLObject is an object-relational mapper. Your database tables are described as classes, and rows are instances of those classes. SQLObject is meant to be easy to use and quick to get started with. SQLObject supports a number of backends: MySQL, PostgreSQL, SQLite, Firebird, Sybase, MSSQL and MaxDB (also known as SAPDB). Where is SQLObject == Site: http://sqlobject.org Development: http://sqlobject.org/devel/ Mailing list: https://lists.sourceforge.net/mailman/listinfo/sqlobject-discuss Archives: http://news.gmane.org/gmane.comp.python.sqlobject Download: http://pypi.python.org/pypi/SQLObject/0.14.1 News and changes: http://sqlobject.org/News.html What's New == News since 0.14.0 - * A bug was fixed in a subtle case when a per-instance connection is not passed to validators. For a more complete list, please see the news: http://sqlobject.org/News.html Oleg. -- Oleg Broytmanhttp://phd.pp.ru/p...@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. -- http://mail.python.org/mailman/listinfo/python-list
Re: EOF while scanning triple-quoted string literal
On Fri, 15 Oct 2010 17:30:20 +1300, Lawrence D'Oliveiro wrote: In message op.vkfl1i1na8n...@gnudebst, Rhodri James wrote: ... frankly putting arbitrary binary into a literal string is rather asking for something like this to come and bite you. It normally works fine on sensible OSes. What does it have to do with the OS? Surely it's a question of the editor, interpreter and related tools. In the Unix world, which includes OS X, text tools tend to have difficulty with tabs. Or try naming a file with a newline or carriage return in the file name, or a NULL byte. Works fine is not how I would describe it. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: PyCharm
Kai Diefenbach wrote: On 2010-10-13 23:36:31 +0200, Robert H said: Since the new IDE from Jetbrains is out I was wondering if you are using it and what you think about it. It sucks. http://regebro.wordpress.com/2010/10/14/python-ide-code-completion-test Kai You're not serious, this 'blog' only focus on one feature: code completion, test it in a very very light manner ; and conclude that a code completion is crap if you need a hotkey to trigger it. I'm not a being fan of IDE, so nothing related to being fan of it, but you can't just discard them cause you need a hotkey to trigger the completion. Who want's to be bothered by windows popping everytime you type a letter anyway ? JM -- http://mail.python.org/mailman/listinfo/python-list
Re: Does everyone keep getting recruiting emails from google?
On 2010-10-15, Philip Semanchuk phi...@semanchuk.com wrote: On Oct 14, 2010, at 11:49 AM, Daniel Fetchinson wrote: I keep getting recruiting emails from charlesngu...@google.com about working for google as an engineer. [...] FWIW, I got one email from Charles Nguyen and answered with a thanks but no thanks. I have not heard from him again. He's perhaps casting too broad a net but the email I got looked legitimately from Google, judging by the headers. Before I replied to the one I got (from somebody other than Nguyen), I took a pretty close look at the headers, and either it was from somebody inside Google, or it was somebody who did a pretty good job forging headers. I've never met a headhunter who even _knew_ what a header was, so I concluded it was the former. -- Grant Edwards grant.b.edwardsYow! Of course, you at UNDERSTAND about the PLAIDS gmail.comin the SPIN CYCLE -- -- http://mail.python.org/mailman/listinfo/python-list
Re: Boolean value of generators
On 2010-10-14, Paul Rubin no.em...@nospam.invalid wrote: Carl Banks pavlovevide...@gmail.com writes: In general, the only way to test if a generator is empty is to try to consume an item. (It's possible to write an iterator that consumes an item and caches it to be returned on the next next(), and whose boolean status indicates if there's an item left. ...) I remember thinking that Python would be better off if all generators automatically cached an item, That would play havoc with generators that had side effects. Not everybody who writes generators is using them to generate Fibonacci numbers. so you could test for emptiness, look ahead at the next item without consuming it, etc. And what happens when the generator is doing things like executing database transactions? This might have been a good change to make in Python 3.0 (it would have broken compatibility with 2.x) but it's too late now. -- Grant Edwards grant.b.edwardsYow! Excuse me, but didn't at I tell you there's NO HOPE gmail.comfor the survival of OFFSET PRINTING? -- http://mail.python.org/mailman/listinfo/python-list
Re: Boolean value of generators
On Fri, 2010-10-15 at 14:54 +, Grant Edwards wrote: so you could test for emptiness, look ahead at the next item without consuming it, etc. And what happens when the generator is doing things like executing database transactions? You should also add prediction to the caching. This will improve performance even more! -a -- http://mail.python.org/mailman/listinfo/python-list
Re: processing input from multiple files
On 10/15/2010 6:59 AM, Christopher Steele wrote:
Thanks,
The issue with the times is now sorted, however I'm running into a
problem towards the end of the script:
File sortoutsynop2.py, line 131, in module
newline =
message_type+c+str(station_id)+c+newtime+c+lat+c+lon+c+c+-+c+
002 +c+-+c+-+c+str(pressure)+c
TypeError: cannot concatenate 'str' and 'list' objects
I think I can see the issue here, but I'm not entirely sure how to get
around it. Several of my variables change either from one file to the
next or from each line. Time and pressure would be examples of both of
these types.Yet others, such as message_type, are constant. As a
result I have a mixture of both lists and strings. Should I then
create a list of the constant values?
I suggest maintaining a list for each such variable, in order to keep
your code simpler. It won't matter that some lists contain the same
value over and over and over.
(There's a slight possibility it would matter if you're dealing with
massive amounts of data. But that's the kind of problem that you don't
need to solve until you encounter it.)
Some more notes below, interspersed with your code ...
I'm a little confused, I'll send you the script that works for a
single file
Yes! That's a much better approach: figure out how to handle one file,
place the code inside a function that takes the filename as an argument,
and call the function on each file in turn.
and I'll see if I can come up with a more logical way around it.
#!/usr/bin/python
import sys
import os
import re
#foutname = 'test.txt'
#filelist = os.system('ls
fname = datalist_201081813.txt
There's a digit missing from the above filename.
foutname1 = 'prestest.txt'
foutname2 = 'temptest.txt'
foutname3 = 'tempdtest.txt'
foutname4 = 'wspeedtest.txt'
foutname5 = 'winddtest.txt'
time = fname.split('_')[1].split('.')[0]
year = time[:4]
month = time[4:6]
day = time[6:8]
hour = time[-2:]
newtime = year+month+day+'_'+hour+''
c = ','
file1 = open(fname,r)
file2 = open(uk_stations.txt,r)
stations = file2.readlines()
ids=[]
names=[]
lats=[]
lons=[]
for item in stations:
item_list = item.strip().split(',')
ids.append(item_list[0])
names.append(item_list[1])
lats.append(item_list[2])
lons.append(item_list[3])
st = file1.readlines()
print st
data=[item[:item.find(' 333 ')] for item in st]
I still think there's a problem in the above statement. In the data file
you provided in a previous message, some lines lack the ' 333 '
substring. In such lines, the find() method will return -1, which (I
think) is not what you want. Ex:
item = '1 2 333 4'
item[:item.find(' 333 ')]
'1 2'
item = '1 2 4'
item[:item.find(' 333 ')]
'1 2 '
Note that the last digit, 4, gets dropped. I *think* you want
something like this:
data = []
posn = item.find(' 333 ')
if posn != -1:
data.append(item[:posn])
else:
data.append(...some other value...)
#data=st[split:]
print data
pres_out = ''
temp_out = ''
dtemp_out = ''
dir_out = ''
speed_out = ''
for line in data:
elements=line.split(' ')
Do you really want to specify a SPACE character argument to split()?
'aaa bbbccc'.split(' ')
['aaa', 'bbb', '', '', '', 'ccc']
'aaa bbbccc'.split()
['aaa', 'bbb', 'ccc']
station_id = elements[0]
try:
index = ids.index(station_id)
lat = lats[index]
lon = lons[index]
message_type = 'blah'
except:
It's bad form to use a bare except, which defines a code block to be
executed if *anything* does wrong. You should specify what you're
expecting to go wrong:
except IndexError:
print 'Station ID',station_id,'not in list!'
lat = lon = 'NaN'
message_type = 'Bad_station_id'
try:
temp = [item for item in elements if item.startswith('1')][0]
temperature = float(temp[2:])/10
sign = temp[1]
if sign == 1:
temperature=-temperature
except:
temperature='NaN'
What are expecting to go wrong (i.e. what exception might occur) in the
above try/except code?
try:
dtemp = [item for item in elements if item.startswith('2')][0]
dtemperature = float(dtemp[2:])/10
sign = dtemp[1]
if sign == 1:
dtemperature=-dtemperature
except:
detemperature='NaN'
try:
press = [item for item in elements[2:] if item.startswith('4')][0]
if press[1]=='9':
pressure = float(press[1:])/10
else:
pressure = float(press[1:])/10+1000
except:
pressure = 'NaN'
try:
wind = elements[elements.index(temp)-1]
direction = float(wind[1:3])*10
speed = float(wind[3:])*0.51444
except:
direction=speed='NaN'
newline =
message_type+c+str(station_id)+c+newtime+c+lat+c+lon+c+'-'+c+'002'+c+'-'+c+'-'+c+str(pressure)+c
Try
open() throws permission error and I don't get why
I'm trying to write a script to read e-mail over stdin, extract attachments, and distribute them around the file system based on the incoming e-mail address. Everything works until I actually try writing interesting file system locations. I've established, through logging, that postfix runs my script with UID nobody GID nobody. The directory I'm attempting to write has permissions 0770. The directory's group is not nobody, but the user nobody is a member of the relevant group. Nevertheless, python throws an IOError when I try to create a file there. The issue seems to be isolated to python. I can run: sudo -u nobody touch DIRECTORY_IN_QUESTION/test and that creates a file. Any ideas? -- http://mail.python.org/mailman/listinfo/python-list
Re: Boolean value of generators
On 2010-10-15, Albert Hopkins mar...@letterboxes.org wrote: On Fri, 2010-10-15 at 14:54 +, Grant Edwards wrote: so you could test for emptiness, look ahead at the next item without consuming it, etc. And what happens when the generator is doing things like executing database transactions? You should also add prediction to the caching. This will improve performance even more! Then I could write a generator that reports stock prices and... -- Grant Edwards grant.b.edwardsYow! Zippy's brain cells at are straining to bridge gmail.comsynapses ... -- http://mail.python.org/mailman/listinfo/python-list
Re: Boolean value of generators
On 10/14/2010 2:21 PM, Cameron Simpson wrote: On 14Oct2010 14:13, Tim Chasepython.l...@tim.thechases.com wrote: | On 10/14/10 12:53, Paul Rubin wrote: |Carl Bankspavlovevide...@gmail.com writes: |In general, the only way to test if a generator is empty is to try to |consume an item. (It's possible to write an iterator that consumes an |item and caches it to be returned on the next next(), and whose |boolean status indicates if there's an item left. ...) | |I remember thinking that Python would be better off if all generators |automatically cached an item, so you could test for emptiness, look |ahead at the next item without consuming it, etc. This might have been |a good change to make in Python 3.0 (it would have broken compatibility |with 2.x) but it's too late now. | | Generators can do dangerous things...I'm not sure I'd *want* to have | Python implicitly cache generators without an explicit wrapper to | request it: [... damaging counter example ...] +1 to this. Speaking for myself, I would _not_ want a generator to commence execution unless I overtly iterate over it. This issue has come up before in I/O systems, where someone may want to test for EOF. In Pascal, there was a predicate to test files for EOF. This didn't work out well for interactive or network input, since testing for EOF had to block until input appeared. The general solution today is to return an EOF token and/or fail a read at EOF. Reading ahead leads to deadlocks. The same logic applies to generators. A generator may be getting data from some source that can block. So an EOF test would lead to trouble. John Nagle -- http://mail.python.org/mailman/listinfo/python-list
Re: open() throws permission error and I don't get why
On Fri, 15 Oct 2010 10:52:53 -0500, Andy Theuninck wrote: I'm trying to write a script to read e-mail over stdin, extract attachments, and distribute them around the file system based on the incoming e-mail address. Everything works until I actually try writing interesting file system locations. I've established, through logging, that postfix runs my script with UID nobody GID nobody. The directory I'm attempting to write has permissions 0770. The directory's group is not nobody, but the user nobody is a member of the relevant group. Nevertheless, python throws an IOError when I try to create a file there. The issue seems to be isolated to python. I can run: sudo -u nobody touch DIRECTORY_IN_QUESTION/test and that creates a file. Any ideas? I suspect that postfix is only setting the UID and the (primary) GID, but not the supplementary GIDs. In which case, it doesn't matter whether nobody is a member of the group. The /etc/group file (or wherever your system gets data for getgrent(), which might include NIS, LDAP, etc) only matters insofar as programs pay attention to it. Typically, programs which perform a login use the UID and GID from /etc/password to set the UID and GID (via setuid() and setgid()), and the information from /etc/group to set the supplementary GIDs (via initgroups() or setgroups()). The permission checks performed by the kernel use the EUID, EGID and/or supplementary GIDs. They do not read /etc/passwd, /etc/group or similar. OTOH, sudo probably /is/ setting the supplementary GIDs, which would explain why sudo...touch works. If my suspicions are correct, you may be able to get around the problem by executing the script via sg (provided that it's installed; it's part of the shadow package; sg itself should be a symlink to newgrp). -- http://mail.python.org/mailman/listinfo/python-list
Re: EOF while scanning triple-quoted string literal
On Oct 14, 10:30 pm, Lawrence D'Oliveiro l...@geek- central.gen.new_zealand wrote: In message op.vkfl1i1na8n...@gnudebst, Rhodri James wrote: ... frankly putting arbitrary binary into a literal string is rather asking for something like this to come and bite you. It normally works fine on sensible OSes. Which OSes would those be? It doesn't work in Linux: $ python -c print 'print \'hello\0world\'' test.py $ cat test.py print 'helloworld' $ python test.py File test.py, line 1 print 'hello ^ SyntaxError: EOL while scanning string literal Cheers, Ian -- http://mail.python.org/mailman/listinfo/python-list
Re: EOF while scanning triple-quoted string literal
On 2010-10-15, Steven D'Aprano st...@remove-this-cybersource.com.au wrote: In the Unix world, which includes OS X, text tools tend to have difficulty with tabs. Or try naming a file with a newline or carriage return in the file name, or a NULL byte. How do you create a file with a name that contains a NULL byte? -- Grant Edwards grant.b.edwardsYow! Catsup and Mustard all at over the place! It's the gmail.comHuman Hamburger! -- http://mail.python.org/mailman/listinfo/python-list
Re: open() throws permission error and I don't get why
I suspect that postfix is only setting the UID and the (primary) GID, but not the supplementary GIDs. In which case, it doesn't matter whether nobody is a member of the group. That does seem like a good explanation. I guess I'll have to re-think my approach a bit. sg sounds like it would give me a different group, but multiple GIDs was really what I was looking for. Back to the drawing board... Thanks for the explanation though. -- http://mail.python.org/mailman/listinfo/python-list
Re: Weird try-except vs if behavior
On 10/15/2010 3:11 AM, Ryan Kelly wrote: On Thu, 2010-10-14 at 22:43 -0700, James Matthews wrote: Hi, I have this code http://gist.github.com/627687 (I don't like pasting code into the mailing list). Yes, but it makes it harder to discuss the ode and makes the archive that much more useless. It's only a couple of lines, here ya go: def tryway(): try: while True: alist.pop() except IndexError: pass def ifway(): while True: if blist == []: break else: blist.pop() if __name__=='__main__': alist = range(1000) blist = range(1000) from timeit import Timer print Testing Try tr = Timer(tryway(),from __main__ import tryway) print tr.timeit() print Testing If ir = Timer(ifway(),from __main__ import ifway) print ir.timeit() I am wondering why the try except is taking longer. I assume that if the IF statement checks every iteration of the loop (1000 times) shouldn't it be slower? You're not measuring what you think. The way you've structured your code, alist and blist are module globals. Timeit will call the tryway function one million times in a loop. The first time it works as you expect, popping each element off of alist in turn. On the other 99 executions, alist is already empty (having been emptied by the first execution) and the IndexError is raised immediately. Likewise for the execution of ifway. So what you're really measuring is the overhead of catching an exception versus testing a condition for an *empty* list - no surprise that the conditional version is faster! If you re-initialise the list at the beginning of each function, the try-based version is faster as you expect. I.e.: def tryway(): alist = range(1000) try: while True: alist.pop() except IndexError: pass I get the following times for a thousand iterations (not waiting around for a million!) Testing Try 0.224129915237 Testing If 0.300312995911 Excellent answer, nothing to add. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: EOF while scanning triple-quoted string literal
On Fri, 15 Oct 2010 17:02:07 +, Grant Edwards wrote: On 2010-10-15, Steven D'Aprano st...@remove-this-cybersource.com.au wrote: In the Unix world, which includes OS X, text tools tend to have difficulty with tabs. Or try naming a file with a newline or carriage return in the file name, or a NULL byte. How do you create a file with a name that contains a NULL byte? Use a language or program that doesn't use null-terminated strings. Its quite easy in many BASICs, which often delimit strings by preceeding it with a with a byte count, and you hit Ctrl-SPACE by accident -- martin@ | Martin Gregorie gregorie. | Essex, UK org | -- http://mail.python.org/mailman/listinfo/python-list
parse xml
I have xml file: ?xml version=1.1 encoding=UTF-8? root n5/n /root I want to get the value of n (= 5) inside my python program, I'm doing this: import xml.dom.minidom from xml.dom.minidom import Node doc = xml.dom.minidom.parseString(boolean_width.xml) n = doc.getElementsByTagName(root)[0].firstChild.nodeValue.strip() print n and it is failed. How to get the value? Please, help. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to display unicode char in Windows
On 10/15/2010 4:57 AM, hiral wrote: Hi, I tried... code # coding: latin-1 print ** oo = ö print char=%s % oo print ** /code but it is not printing ö char; any idea? Thank you. -Hiral Unicode output to Windows consoles has been broken since 2007. See http://bugs.python.org/issue1602 Surprisingly, it actually worked with Python 2.5 and Windows 2000, if you changed the Windows console encoding to chcp 65001. If you try that with Python 2.6 and Windows 7, you get LookupError: unknown encoding: cp65001, because cp65001 isn't in Python's encoding tables. John Nagle -- http://mail.python.org/mailman/listinfo/python-list
Re: PyCharm
On Oct 14, 8:49 pm, alex23 wuwe...@gmail.com wrote: Jeffrey Gaynor jgay...@ncsa.uiuc.edu wrote: Certainly give it a shot. The only other IDE I found that was remotely close to it was Komodo which costs a lot more (Jetbrains is offering a 50% off coupon as a promotional offer for a while.) I recently tried out PyCharm in anger after something (I forget what) in Komodo was bothering me. In Komodo's defence, it supports Perl, PHP, Python Ruby, two of which I use daily, so replacing it would require my buying two IDEs: PyCharm PHPStorm. It would just be a damn sight easier if I didn't have to suffer under PHP :( Yes, it is the multi-language support in Komodo that is pushing me that way. Bob -- http://mail.python.org/mailman/listinfo/python-list
Re: PyCharm
On Oct 14, 4:16 pm, Jeffrey Gaynor jgay...@ncsa.uiuc.edu wrote: Yip. I'm using it and for the most part like it. But... I used their Java IDE for years (it totally rocks, highly recommended), so I it is very comfortable to use PyCharm. One thing that bugs me in refactoring though is that renaming a method or variable does not necessarily work. It's supposed to track down all references and correctly change them, but it tends to be hit or miss. No problem though, since I just do a search of the files in question and do it manually. Still, the Java refactoring engine works very well indeed and id one of their major selling points. Code completion works, you can specify different Python versions (helpful) and there is Django support. The debugger, though I have only had limited use for it, does seem to work well too. Certainly give it a shot. The only other IDE I found that was remotely close to it was Komodo which costs a lot more (Jetbrains is offering a 50% off coupon as a promotional offer for a while.) Hope this helps... Thanks for the answer. -- http://mail.python.org/mailman/listinfo/python-list
Re: EOF while scanning triple-quoted string literal
On 2010-10-15, Martin Gregorie mar...@address-in-sig.invalid wrote: On Fri, 15 Oct 2010 17:02:07 +, Grant Edwards wrote: On 2010-10-15, Steven D'Aprano st...@remove-this-cybersource.com.au wrote: In the Unix world, which includes OS X, text tools tend to have difficulty with tabs. Or try naming a file with a newline or carriage return in the file name, or a NULL byte. How do you create a file with a name that contains a NULL byte? Use a language or program that doesn't use null-terminated strings. Its quite easy in many BASICs, which often delimit strings by preceeding it with a with a byte count, and you hit Ctrl-SPACE by accident I don't see what the in-program string representation has to do with it. The Unix system calls that create files only accept NULL terminated strings for the path parameter. Are you saying that there are BASIC implementations for Unix that create Unix files by directly accessing the disk rather than using the Unix system calls? -- Grant Edwards grant.b.edwardsYow! I'm sitting on my at SPEED QUEEN ... To me, gmail.comit's ENJOYABLE ... I'm WARM ... I'm VIBRATORY ... -- http://mail.python.org/mailman/listinfo/python-list
Re: How to display unicode char in Windows
hiral hiralsmaill...@gmail.com wrote in message news:90b62600-a0a4-47d5-bb6f-a3ae14cf6...@9g2000prn.googlegroups.com... Hi, I tried... code # coding: latin-1 print ** oo = ö print char=%s % oo print ** /code but it is not printing ö char; any idea? 1) Make sure you save your source in the encoding declared, or change the encoding to match. Any encoding that supports ö will work. 2) Use Unicode strings. 3) Only print characters your terminal supports, or you will get a UnicodeEncoding error. Below works in PythonWin GUI (utf-8 encoding) and US-Windows console (cp437 encoding). code # coding: latin-1 print u** oo = uö print uchar=%s % oo print u** /code Coding line declares *source* encoding, so Python can *decode* characters contained in the source. print will *encode* characters to the terminal encoding, if known. -Mark -- http://mail.python.org/mailman/listinfo/python-list
Re: parse xml
On Fri, 15 Oct 2010 10:49:18 -0700 (PDT) kostia kostya.demc...@gmail.com wrote: I have xml file: ?xml version=1.1 encoding=UTF-8? root n5/n /root I want to get the value of n (= 5) inside my python program, I'm doing this: import xml.dom.minidom from xml.dom.minidom import Node doc = xml.dom.minidom.parseString(boolean_width.xml) n = doc.getElementsByTagName(root)[0].firstChild.nodeValue.strip() print n and it is failed. [snip] How? What's the error message? -- To reach me via email, replace INVALID with the country code of my home country. But if you spam me, I'll be one sour Kraut. -- http://mail.python.org/mailman/listinfo/python-list
Re: parse xml
On 10/15/2010 11:19 PM, kostia wrote:
I have xml file:
?xml version=1.1 encoding=UTF-8?
root
n5/n
/root
I want to get the value of n (= 5) inside my python program, I'm
doing this:
import xml.dom.minidom
from xml.dom.minidom import Node
doc = xml.dom.minidom.parseString(boolean_width.xml)
Use parse to parse a file. doc = parse('boolean_width.xml')
n = doc.getElementsByTagName(root)[0].firstChild.nodeValue.strip()
print n
and it is failed. How to get the value? Please, help.
Here's a complete program:
import xml.dom.minidom
from xml.dom.minidom import Node, parse
doc = parse('boolean_width.xml')
my_node_list = doc.getElementsByTagName(n)
my_n_node = my_node_list[0]
my_child = my_n_node.firstChild
my_text = my_child.data
--
With warm regards,
Sudheer. S
Personal home page - http://sudheer.net | Tech Chorus -
http://techchorus.net
Web and IT services - http://binaryvibes.co.in
--
http://mail.python.org/mailman/listinfo/python-list
Re: parse xml
this is wrong xml.dom.minidom.parseString(boolean_width.xml) ... if u r parsing from String use string variable as argument or use parse only if parsing from file On Sat, Oct 16, 2010 at 12:07 AM, Andreas Waldenburger use...@geekmail.invalid wrote: On Fri, 15 Oct 2010 10:49:18 -0700 (PDT) kostia kostya.demc...@gmail.com wrote: I have xml file: ?xml version=1.1 encoding=UTF-8? root n5/n /root I want to get the value of n (= 5) inside my python program, I'm doing this: import xml.dom.minidom from xml.dom.minidom import Node doc = xml.dom.minidom.parseString(boolean_width.xml) n = doc.getElementsByTagName(root)[0].firstChild.nodeValue.strip() print n and it is failed. [snip] How? What's the error message? -- To reach me via email, replace INVALID with the country code of my home country. But if you spam me, I'll be one sour Kraut. -- http://mail.python.org/mailman/listinfo/python-list -- Nitin Pawar -- http://mail.python.org/mailman/listinfo/python-list
Re: parse xml
On 15/10/2010 18:49, kostia wrote: I have xml file: ?xml version=1.1 encoding=UTF-8? root n5/n /root I want to get the value of n (= 5) inside my python program, I'm doing this: import xml.dom.minidom from xml.dom.minidom import Node doc = xml.dom.minidom.parseString(boolean_width.xml) n = doc.getElementsByTagName(root)[0].firstChild.nodeValue.strip() print n and it is failed. How to get the value? Please, help. The 'parseString' method does what it says, it parses a string. You're giving it the string boolean_width.xml, so that's what it's parsing. If you want to parse a file then use the 'parse' method. -- http://mail.python.org/mailman/listinfo/python-list
Re: EOF while scanning triple-quoted string literal
On 2010-10-15, Grant Edwards inva...@invalid.invalid wrote: On 2010-10-15, Steven D'Aprano st...@remove-this-cybersource.com.au wrote: In the Unix world, which includes OS X, text tools tend to have difficulty with tabs. Or try naming a file with a newline or carriage return in the file name, or a NULL byte. How do you create a file with a name that contains a NULL byte? So far as I know, in canonical Unix, you don't -- the syscalls all work with something like C strings under the hood, meaning that no matter what path name you send, the first null byte actually terminates it. -s -- Copyright 2010, all wrongs reversed. Peter Seebach / usenet-nos...@seebs.net http://www.seebs.net/log/ -- lawsuits, religion, and funny pictures http://en.wikipedia.org/wiki/Fair_Game_(Scientology) -- get educated! I am not speaking for my employer, although they do rent some of my opinions. -- http://mail.python.org/mailman/listinfo/python-list
Re: EOF while scanning triple-quoted string literal
On 2010-10-15, Seebs usenet-nos...@seebs.net wrote: On 2010-10-15, Grant Edwards inva...@invalid.invalid wrote: On 2010-10-15, Steven D'Aprano st...@remove-this-cybersource.com.au wrote: In the Unix world, which includes OS X, text tools tend to have difficulty with tabs. Or try naming a file with a newline or carriage return in the file name, or a NULL byte. How do you create a file with a name that contains a NULL byte? So far as I know, in canonical Unix, you don't -- the syscalls all work with something like C strings under the hood, meaning that no matter what path name you send, the first null byte actually terminates it. Yes, all of the Unix syscalls use NULL-terminated path parameters (AKA C strings). What I don't know is whether the underlying filesystem code also uses NULL-terminated strings for filenames or if they have explicit lengths. If the latter, there might be some way to bypass the normal Unix syscalls and actually create a file with a NULL in its name -- a file that then couldn't be accessed via the normal Unix system calls. My _guess_ is that the underlying filesystem code in most all Unices also uses NULL-terminated strings, but I haven't looked yet. -- Grant Edwards grant.b.edwardsYow! What UNIVERSE is this, at please?? gmail.com -- http://mail.python.org/mailman/listinfo/python-list
Re: EOF while scanning triple-quoted string literal
On Fri, 15 Oct 2010 18:14:13 +, Grant Edwards wrote: On 2010-10-15, Martin Gregorie mar...@address-in-sig.invalid wrote: On Fri, 15 Oct 2010 17:02:07 +, Grant Edwards wrote: On 2010-10-15, Steven D'Aprano st...@remove-this-cybersource.com.au wrote: In the Unix world, which includes OS X, text tools tend to have difficulty with tabs. Or try naming a file with a newline or carriage return in the file name, or a NULL byte. How do you create a file with a name that contains a NULL byte? Use a language or program that doesn't use null-terminated strings. Its quite easy in many BASICs, which often delimit strings by preceeding it with a with a byte count, and you hit Ctrl-SPACE by accident I don't see what the in-program string representation has to do with it. The Unix system calls that create files only accept NULL terminated strings for the path parameter. Well, obviously you can't have null in a filename if the program is using null-terminated strings. Are you saying that there are BASIC implementations for Unix that create Unix files by directly accessing the disk rather than using the Unix system calls? I'm saying that the only BASIC implementations I've looked at the guts of have used count-delimited strings. None were on *nixen but its a safe bet that if they were ported to a UNIX they'd retain their count-delimited nature. Another language that will certainly do this is COBOL, which only uses fixed length, and therefore undelimited, strings. The point I'm making is that in both fixed length and counted string representations you can put any character value at all into the string unless whatever mechanism you're using to read in the values recognises something, i.e. TAB, CR, LF, CRLF as a delimiter, and even then the program can generate a string containing arbitrary gibberish. If you then use the string as a file name you can end up with a file that can't be accessed or deleted if the name flouts the OS's file naming conventions. I've done it in the past with BASIC programs and finger trouble under FLEX09 and CP/M. In both cases I had to use a disk editor to fix the file name before the file could be deleted or accessed. -- martin@ | Martin Gregorie gregorie. | Essex, UK org | -- http://mail.python.org/mailman/listinfo/python-list
Re: parse xml
Thank you all! -- http://mail.python.org/mailman/listinfo/python-list
Windows binary stdin goes EOF after \x1a character
I am writing a Windows program in Python 3.1.2 that reads binary data
from stdin. Whenever it hits a \x1a character, stdin goes EOF and no
more data can be read. A short program that exhibits this problem is:
#listing of main.pyw
import sys
def go():
bb=sys.stdin.buffer.raw.read(1)
print(bb,file=sys.stderr)
sys.stderr.flush()
go()
go()
The program is run as pythonw.exe main.pyw
When fed with a block of bytes equivalent to bytes(range(1,255)),
the following output is observed...
b'\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f
\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19'
b''
...and according to the documentation, the empty bytes object means
EOF.
I wrote an equivalent program in C++ using the win32 ReadFile() call,
and it read all 255 bytes just fine. What am I doing wrong with the
python code?
I am using Erlang to launch the Python program as a subprocess. The
Erlang fragment that launches this and sends the data is:
Port=open_port( {spawn_executable, c:/Python31/pythonw.exe},
[{args, [c:/iotest/main.pyw]}]),
Port ! {self(),{command,lists:seq(1,255)}},
Thanks,
Dan.
--
http://mail.python.org/mailman/listinfo/python-list
Re: EOF while scanning triple-quoted string literal
On 2010-10-15, Grant Edwards inva...@invalid.invalid wrote: Yes, all of the Unix syscalls use NULL-terminated path parameters (AKA C strings). What I don't know is whether the underlying filesystem code also uses NULL-terminated strings for filenames or if they have explicit lengths. If the latter, there might be some way to bypass the normal Unix syscalls and actually create a file with a NULL in its name -- a file that then couldn't be accessed via the normal Unix system calls. My _guess_ is that the underlying filesystem code in most all Unices also uses NULL-terminated strings, but I haven't looked yet. There's some dire magic there. The classic V7 or so filesystem had 16-byte file names which were null terminated unless they were 16 characters, in which case they weren't but were still only 16 characters. Apart from that, though, so far as I know everything is always null terminated. The weird special case is slashes; you can never have a slash in a file name, but at least one NFS implementation was able to create file names containing slashes, and if you had a Mac client (where slash was valid in file names), it could then create files with names that you could never use on the Unix side, because the path resolution code kept trying to find directories instead. This was, worse yet, common, because so many people used mm/dd/yy in file names! Later implementations changed to silently translating between colons and slashes. (I think this still happened under the hood in at least some OS X, because the HFS filesystem really uses colons somewhere down in there.) ... But so far as I know, there's never been a Unix-type system where it was actually possible to get a null byte into a file name. Spaces, newlines, sure. Slashes, under rare and buggy circumstances. But I've never heard of a null byte in a file name. -s -- Copyright 2010, all wrongs reversed. Peter Seebach / usenet-nos...@seebs.net http://www.seebs.net/log/ -- lawsuits, religion, and funny pictures http://en.wikipedia.org/wiki/Fair_Game_(Scientology) -- get educated! I am not speaking for my employer, although they do rent some of my opinions. -- http://mail.python.org/mailman/listinfo/python-list
Re: Does everyone keep getting recruiting emails from google?
My understanding is that Google headhunters are contractors and not regular Google employees. This hired gun approach to recruiting may lead to the use of unusual methods in order to meet their requirements. -- http://mail.python.org/mailman/listinfo/python-list
Re: Windows binary stdin goes EOF after \x1a character
On 2010-10-15, Dan daniel.goert...@gmail.com wrote: I am writing a Windows program in Python 3.1.2 that reads binary data from stdin. Whenever it hits a \x1a character, stdin goes EOF and no more data can be read. A short program that exhibits this problem is: stdin is not in binary mode. Try invoking with -u option. -- Neil Cerutti -- http://mail.python.org/mailman/listinfo/python-list
Re: EOF while scanning triple-quoted string literal
On 2010-10-15, Grant Edwards inva...@invalid.invalid wrote: How do you create a [Unix] file with a name that contains a NULL byte? On 2010-10-15, Seebs usenet-nos...@seebs.net wrote: So far as I know, in canonical Unix, you don't -- the syscalls all work with something like C strings under the hood, meaning that no matter what path name you send, the first null byte actually terminates it. In article i9a84m$rp...@reader1.panix.com Grant Edwards inva...@invalid.invalid wrote: Yes, all of the Unix syscalls use NULL-terminated path parameters (AKA C strings). What I don't know is whether the underlying filesystem code also uses NULL-terminated strings for filenames or if they have explicit lengths. If the latter, there might be some way to bypass the normal Unix syscalls and actually create a file with a NULL in its name -- a file that then couldn't be accessed via the normal Unix system calls. My _guess_ is that the underlying filesystem code in most all Unices also uses NULL-terminated strings, but I haven't looked yet. Multiple common on-disk formats (BSD's UFS variants and Linux's EXTs, for instance) use counted strings, so it is possible -- via disk corruption or similar -- to get impossible file names (those containing either an embedded NUL or an embedded '/'). More notoriously, earlier versions of NFS could create files with embedded slashes when serving non-Unix clients. These were easily removed with the same non-Unix client, but not on the server! :-) None of this has anything to do with the original problem, in which a triple-quoted string is left to contain arbitrary binary data (up to, of course, the closing triple-quote). Should that arbitrary binary data itself happen to include a triple-quote, this trivial encoding technique will fail. (And of course, as others have noted, it fails on some systems that distinguish betwen text and binary file formats in the first place.) This is why using some text-friendly encoding scheme, such as base64, is a good idea. -- In-Real-Life: Chris Torek, Wind River Systems Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W) +1 801 277 2603 email: gmail (figure it out) http://web.torek.net/torek/index.html -- http://mail.python.org/mailman/listinfo/python-list
Re: Windows binary stdin goes EOF after \x1a character
On Oct 15, 2:42 pm, Neil Cerutti ne...@norwich.edu wrote: On 2010-10-15, Dan daniel.goert...@gmail.com wrote: I am writing a Windows program in Python 3.1.2 that reads binary data from stdin. Whenever it hits a \x1a character, stdin goes EOF and no more data can be read. A short program that exhibits this problem is: stdin is not in binary mode. Try invoking with -u option. -- Neil Cerutti That worked, thanks! Dan. -- http://mail.python.org/mailman/listinfo/python-list
Re: EOF while scanning triple-quoted string literal
On 2010-10-15, Martin Gregorie mar...@address-in-sig.invalid wrote: On 2010-10-15, Martin Gregorie mar...@address-in-sig.invalid wrote: On Fri, 15 Oct 2010 17:02:07 +, Grant Edwards wrote: On 2010-10-15, Steven D'Aprano st...@remove-this-cybersource.com.au: In the Unix world, which includes OS X, text tools tend to have difficulty with tabs. Or try naming a file with a newline or carriage return in the file name, or a NULL byte. How do you create a file with a name that contains a NULL byte? Use a language or program that doesn't use null-terminated strings. Its quite easy in many BASICs, [...] I don't see what the in-program string representation has to do with it. The Unix system calls that create files only accept NULL terminated strings for the path parameter. Well, obviously you can't have null in a filename if the program is using null-terminated strings. Obviously. Just as obviously, you can't have a null in a filename if the OS filesystem API uses null-terminated strings -- which the Linux filesystem API does. I just verified that by looking at the kernel sources -- I can post the relevent code if you like. I'm pretty sure all the other Unices are the same. I've got BSD sources laying around somewhere... Are you saying that there are BASIC implementations for Unix that create Unix files by directly accessing the disk rather than using the Unix system calls? I'm saying that the only BASIC implementations I've looked at the guts of have used count-delimited strings. None were on *nixen but its a safe bet that if they were ported to a UNIX they'd retain their count-delimited nature. And I'm saying _that_doesn't_matter_. The _OS_ uses NULL-terminated strings. You can use a language the represents strings as braille images encoded as in-memory PNG files if you want. That still doesn't let you create a Unix file whose name contains a NULL byte. Another language that will certainly do this is COBOL, which only uses fixed length, and therefore undelimited, strings. Again, what difference does it make? If the OS uses null-terminated strings for filenames, what difference does it make how the user-space program represents filenames internally? The point I'm making is that in both fixed length and counted string representations you can put any character value at all into the string unless whatever mechanism you're using to read in the values recognises something, i.e. TAB, CR, LF, CRLF as a delimiter, and even then the program can generate a string containing arbitrary gibberish. I don't care how the program represents strings. The OS doesn't care. The filesystem doesn't care. Please explain how to pass a filename containing a NULL byte to a Unix syscall like creat() or open(). You don't even have to use the C library API -- feel free to use the real syscall API for whatever Unix on whatever architecture you want. If you then use the string as a file name you can end up with a file that can't be accessed or deleted if the name flouts the OS's file naming conventions. I've done it in the past with BASIC programs and finger trouble under FLEX09 and CP/M. In both cases I had to use a disk editor to fix the file name before the file could be deleted or accessed. We're talking about Unix. We're not talking about CP/M, DOS, RSX-11m, Apple-SOS, etc. -- Grant Edwards grant.b.edwardsYow! I put aside my copy at of BOWLING WORLD and gmail.comthink about GUN CONTROL legislation... -- http://mail.python.org/mailman/listinfo/python-list
Re: EOF while scanning triple-quoted string literal
On 2010-10-15, Seebs usenet-nos...@seebs.net wrote: On 2010-10-15, Grant Edwards inva...@invalid.invalid wrote: Yes, all of the Unix syscalls use NULL-terminated path parameters (AKA C strings). What I don't know is whether the underlying filesystem code also uses NULL-terminated strings for filenames or if they have explicit lengths. If the latter, there might be some way to bypass the normal Unix syscalls and actually create a file with a NULL in its name -- a file that then couldn't be accessed via the normal Unix system calls. My _guess_ is that the underlying filesystem code in most all Unices also uses NULL-terminated strings, but I haven't looked yet. There's some dire magic there. The classic V7 or so filesystem had 16-byte file names which were null terminated unless they were 16 characters, in which case they weren't but were still only 16 characters. Apart from that, though, so far as I know everything is always null terminated. I've just verfied in the Linux sources that filenames passed to linux syscall API (as opposed to the C library API) are indeed null terminated. Even if they're not stored as null-terminated strings in the actual filesystem data-structures, there's no way to get a null byte in there using standard syscalls. I have found a few places in the filesystem code where there's a structure field that seems to be a name length, but it's not obvious at first glance if that's a file name. The weird special case is slashes; you can never have a slash in a file name, but at least one NFS implementation was able to create file names containing slashes, and if you had a Mac client (where slash was valid in file names), it could then create files with names that you could never use on the Unix side, because the path resolution code kept trying to find directories instead. Fun! This was, worse yet, common, because so many people used mm/dd/yy in file names! Later implementations changed to silently translating between colons and slashes. (I think this still happened under the hood in at least some OS X, because the HFS filesystem really uses colons somewhere down in there.) ... But so far as I know, there's never been a Unix-type system where it was actually possible to get a null byte into a file name. Spaces, newlines, sure. And even more fun, backspaces and ANSI escape sequences. :) Back in the day when everybody was sitting at a terminal (or at least an xterm), you could confuse somebody for days with judicious use of filenames containing escape sequences. Not that I'd ever do such a thing. Slashes, under rare and buggy circumstances. But I've never heard of a null byte in a file name. Nor I, which is why I was confused by the statement that in the Unix world a lot of programs misbehaved when presented with files whose names contained a null byte. -- Grant Edwards grant.b.edwardsYow! I want my nose in at lights! gmail.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Windows binary stdin goes EOF after \x1a character
On 15/10/2010 20:28, Dan wrote:
I am writing a Windows program in Python 3.1.2 that reads binary data
from stdin. Whenever it hits a \x1a character, stdin goes EOF and no
more data can be read. A short program that exhibits this problem is:
#listing of main.pyw
import sys
def go():
bb=sys.stdin.buffer.raw.read(1)
print(bb,file=sys.stderr)
sys.stderr.flush()
go()
go()
The program is run as pythonw.exe main.pyw
When fed with a block of bytes equivalent to bytes(range(1,255)),
the following output is observed...
b'\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f
\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19'
b''
...and according to the documentation, the empty bytes object means
EOF.
I wrote an equivalent program in C++ using the win32 ReadFile() call,
and it read all 255 bytes just fine. What am I doing wrong with the
python code?
I am using Erlang to launch the Python program as a subprocess. The
Erlang fragment that launches this and sends the data is:
Port=open_port( {spawn_executable, c:/Python31/pythonw.exe},
[{args, [c:/iotest/main.pyw]}]),
Port ! {self(),{command,lists:seq(1,255)}},
By default, Python opens stdin in buffered text mode in which '\x1A'
marks the end of the text. Try adding the -u option (unbuffered) to
Python's command line:
pythonw.exe -u main.pyw
--
http://mail.python.org/mailman/listinfo/python-list
Re: EOF while scanning triple-quoted string literal
On 2010-10-15, Chris Torek nos...@torek.net wrote: On 2010-10-15, Grant Edwards inva...@invalid.invalid wrote: How do you create a [Unix] file with a name that contains a NULL byte? On 2010-10-15, Seebs usenet-nos...@seebs.net wrote: So far as I know, in canonical Unix, you don't -- the syscalls all work with something like C strings under the hood, meaning that no matter what path name you send, the first null byte actually terminates it. In article i9a84m$rp...@reader1.panix.com Grant Edwards inva...@invalid.invalid wrote: Yes, all of the Unix syscalls use NULL-terminated path parameters (AKA C strings). What I don't know is whether the underlying filesystem code also uses NULL-terminated strings for filenames or if they have explicit lengths. If the latter, there might be some way to bypass the normal Unix syscalls and actually create a file with a NULL in its name -- a file that then couldn't be accessed via the normal Unix system calls. My _guess_ is that the underlying filesystem code in most all Unices also uses NULL-terminated strings, but I haven't looked yet. Multiple common on-disk formats (BSD's UFS variants and Linux's EXTs, for instance) use counted strings, so it is possible -- via disk corruption or similar -- to get impossible file names (those containing either an embedded NUL or an embedded '/'). That appeared it might be the case after a quick browsing of some of the fs source code, but I wasn't sure. More notoriously, earlier versions of NFS could create files with embedded slashes when serving non-Unix clients. These were easily removed with the same non-Unix client, but not on the server! :-) None of this has anything to do with the original problem, No, we left that track miles back... -- Grant Edwards grant.b.edwardsYow! I'm having an at emotional outburst!! gmail.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Windows binary stdin goes EOF after \x1a character
On 2010-10-15, MRAB pyt...@mrabarnett.plus.com wrote:
I wrote an equivalent program in C++ using the win32
ReadFile() call, and it read all 255 bytes just fine. What am
I doing wrong with the python code?
I am using Erlang to launch the Python program as a subprocess. The
Erlang fragment that launches this and sends the data is:
Port=open_port( {spawn_executable, c:/Python31/pythonw.exe},
[{args, [c:/iotest/main.pyw]}]),
Port ! {self(),{command,lists:seq(1,255)}},
By default, Python opens stdin in buffered text mode in which
'\x1A' marks the end of the text. Try adding the -u option
(unbuffered) to Python's command line:
pythonw.exe -u main.pyw
To expound a bit, using a windows-only module you can switch the
mode of stdin.
import sys
import msvcrt
msvcrt.setmode(sys.stdin.fileno(), os.O_BINARY)
--
Neil Cerutti
--
http://mail.python.org/mailman/listinfo/python-list
PEP 380 - the 'yield from' proposal
Regarding http://www.python.org/dev/peps/pep-0380/, Syntax for Delegating to a Subgenerator: The first call can only be .next(), there's no way to provide an initial value to .send(). That matches common use, but an initial .send() is possible if .next() was called before yield from. So I suggest: RESULT = yield from EXPR [with SEND_FIRST] # default SEND_FIRST=None The code under Formal Semantics uses .throw and .close = None as equivalent to absent attributes, which is not what the textual description says. I think the code should delete values when it no longer has use for them, so they can be garbage-collected as quickly as possible. So the formal semantics would be something like i, snd, yld, throw = iter(EXPR), SEND_FIRST, None, None res = absent = object() # Internal marker, never exposed try: while res is absent: try: yld = (i.next() if snd is None else i.send(snd)) except StopIteration as e: res = e.value else: snd = absent # 'del snd', but that could break 'finally:' while yld is not absent: try: snd = yield yld yld = absent except: del yld if throw is None: # optional statement throw = getattr(i, 'throw', absent) if throw is absent: getattr(i, 'close', bool)() # bool()=dummy raise x = sys.exc_info() try: yld = throw(*x) except StopIteration as e: if e is x[1] or isinstance(x[1], GeneratorExit): raise res = e.value finally: del x finally: del i, snd, throw RESULT = res del res Maybe it's excessive to specify all the 'del's, but I'm thinking the 'with' statement might have destroyed or set to None the 'as foo' variable today if the spec had taken care to specify deletes. -- Hallvard -- http://mail.python.org/mailman/listinfo/python-list
Re: EOF while scanning triple-quoted string literal
On Fri, 15 Oct 2010 19:59:13 +, Grant Edwards wrote: We're talking about Unix. We're not talking about CP/M, DOS, RSX-11m, Apple-SOS, etc. That's just your assumption. Track back up the thread and you'll see that the OP didn't mention an OS. He merely said that he was using zlib, and getting unfortunate results when he handled its output, so he could have been using any OS. Rhodri James assumed that the OS was Windows, but it wasn't until the 6th post that Steven D'Aprano mentioned Unix and null characters. I got sucked into the null trap - sorry - because I actually meant to generalise the discussion into ways of getting a range of unwanted characters into a file name and why its unwise to use a filename without checking it for characters the OS doesn't like before creating a file with it. -- martin@ | Martin Gregorie gregorie. | Essex, UK org | -- http://mail.python.org/mailman/listinfo/python-list
Re: Class-level variables - a scoping issue
Lawrence D'Oliveiro wrote: “Dynamically” is when that “future” becomes the present, so you can see it right in front of you, here, now. But raising an UnboundAttributeError when attempting to read a class attribute that *could*, but hasn't yet, been shadowed by an instance attribute requires taking action *before* the future becomes the present. Unless I've misunderstood what you were suggesting the behaviour should be. -- Greg -- http://mail.python.org/mailman/listinfo/python-list
Re: what happens to Popen()'s parent-side file descriptors?
In message pan.2010.10.15.06.27.02.360...@nowhere.com, Nobody wrote: Another gotcha regarding pipes: the reader only sees EOF once there are no writers, i.e. when the *last* writer closes their end. Been there, been bitten by that. -- http://mail.python.org/mailman/listinfo/python-list
Re: PEP 249 (database api) -- executemany() with iterable?
In message ecdbf6b3-3f99-4b59-b7f8-85bd22f97...@w9g2000prc.googlegroups.com, Steve Howell wrote: On Oct 13, 8:32 pm, Lawrence D'Oliveiro l...@geek-central.gen.new_zealand wrote: In message d2451907-c0d2-4571-b3e1-1e4d4f66a...@a7g2000prb.googlegroups.com, Steve Howell wrote: Bulk-load strategies usually solve one or more of these problems: network latency That’s not an issue. This is a bulk operation, after all. index maintenance during the upload There are usually options to temporarily turn this off. parsing of the SQL Not usually a problem, as far as I can tell. reallocating storage You mean for thr SQL? Again, not usually a problem, as far as I can tell. If you are avoiding network latency and turning off indexes, then you are using some kind of a bulk-load strategy. I thought we were talking about “load data” versus using simple variants of “insert”. At least, that’s what I was talking about. What were you talking about? If you are not concerned about parsing costs or storage churn ... What is this “storage churn” thing, and why are you raising it now? -- http://mail.python.org/mailman/listinfo/python-list
Re: PyCharm
In message mailman.1755.1287150954.29448.python-l...@python.org, Jean- Michel Pichavant wrote: Who want's to be bothered by windows popping everytime you type a letter anyway ? Apparently the Java and Visual Studio crowd are big on this sort of thing. -- http://mail.python.org/mailman/listinfo/python-list
Re: PyCharm
In message mailman.1737.1287096167.29448.python-l...@python.org, Jeffrey Gaynor wrote: One thing that bugs me in refactoring though is that renaming a method or variable does not necessarily work. It's supposed to track down all references and correctly change them, but it tends to be hit or miss. Given Python’s dynamic nature, I think that’s an undecidable problem. -- http://mail.python.org/mailman/listinfo/python-list
Re: what happens to Popen()'s parent-side file descriptors?
In message pan.2010.10.15.06.27.02.360...@nowhere.com, Nobody wrote: Another gotcha regarding pipes: the reader only sees EOF once there are no writers, i.e. when the *last* writer closes their end. In article i9atra$j4...@lust.ihug.co.nz Lawrence D'Oliveiro l...@geek-central.gen.new_zealand wrote: Been there, been bitten by that. Nobody mentioned the techniques of setting close_fds = True and passing a preexec_fn that closes the extra pipe descriptors. You can also use fcntl.fcntl() to set the fcntl.FD_CLOEXEC flag on the underlying file descriptors (this of course requires that you are able to find them). The subprocess module sets FD_CLOEXEC on the pipe it uses to pass back a failure to exec, or even to reach the exec, e.g., due to an exception during preexec_fn. One could argue that perhaps it should set FD_CLOEXEC on the parent's remaining pipe descriptors, once the child is successfully started, if it created them (i.e., if the corresponding arguments were PIPE). In fact, thinking about it now, I *would* argue that. -- In-Real-Life: Chris Torek, Wind River Systems Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W) +1 801 277 2603 email: gmail (figure it out) http://web.torek.net/torek/index.html -- http://mail.python.org/mailman/listinfo/python-list
Re: Scheme as a virtual machine?
namekuseijin namekusei...@gmail.com wrote:
On 14 out, 00:26, Ertugrul Söylemez e...@ertes.de wrote:
BTW, you mentioned symbols ('$', '.' and '='), which are not
syntactic sugar at all. They are just normal functions, for which
it makes sense to be infix. The fact that you sold them as
syntactic sugar or perlisms proves that you have no idea about the
language, so stop crying. Also Python-style significant whitespace
is strictly optional. It's nice though. After all most Haskell
programmers prefer it.
it still makes haskell code scattered with perlisms, be it syntax or
function name... in practice, Haskell code is ridden with such
perlisms and significant whitespace, and infix function application
and more special cases. All of these contribute to a harder to parse
language [...]
So what? The quality of a language isn't measured by the difficulty to
parse it. Haskell has certainly more syntactic special cases than
Scheme, but I don't care, because they are /useful/.
[...] and to less compilers for it.
That's an arbitrary and wrong statement. The reason why there aren't
many Haskell compilers is that Haskell needs a good run-time system and
a lot of algorithms, which you wouldn't need in languages like Scheme
(including typed Scheme), which have a comparably simple type system.
Also you have to deal with laziness, and ideally you would want to write
a smart optimizer. This is easier for other languages.
But what's the matter? GHC is BSD-licensed. Derive your project from
it, if you are, for some reason, not happy with it.
And one as complex and scary beast as gcc... that's the cost of a
very irregular syntax...
What also proves that you have no idea is the fact that there is no
Haskell compiler called 'gcc'. That's the GNU C compiler.
ORLY?
do you understand what a comparison is?
Sure, sure. I'd probably say that, too, in your situation. ;)
Glasgow Haskell Compiler, GHC, and it's by far not the only
one. It's just the one most people use, and there is such a
compiler for all languages.
yeah, there's also some Yale Haskell compiler in some graveyard, last
time I heard...
http://haskell.org/haskellwiki/Implementations
Greets,
Ertugrul
--
nightmare = unsafePerformIO (getWrongWife = sex)
http://ertes.de/
--
http://mail.python.org/mailman/listinfo/python-list
Re: parse xml
First every element represents a node so you have to use value=n.childNodes[0].nodeValue with that you'll have the 5000 2010/10/15, kostia kostya.demc...@gmail.com: I have xml file: ?xml version=1.1 encoding=UTF-8? root n5/n /root I want to get the value of n (= 5) inside my python program, I'm doing this: import xml.dom.minidom from xml.dom.minidom import Node doc = xml.dom.minidom.parseString(boolean_width.xml) n = doc.getElementsByTagName(root)[0].firstChild.nodeValue.strip() print n and it is failed. How to get the value? Please, help. -- http://mail.python.org/mailman/listinfo/python-list -- Enviado desde mi dispositivo móvil Diego I. Hidalgo D. -- http://mail.python.org/mailman/listinfo/python-list
Re: EOF while scanning triple-quoted string literal
On 2010-10-15, Martin Gregorie mar...@address-in-sig.invalid wrote: On Fri, 15 Oct 2010 19:59:13 +, Grant Edwards wrote: We're talking about Unix. We're not talking about CP/M, DOS, RSX-11m, Apple-SOS, etc. That's just your assumption. If you go back and look at my original posting in this thread, here's what I was replying to: In the Unix world, which includes OS X, text tools tend to have difficulty with tabs. Or try naming a file with a newline or carriage return in the file name, or a NULL byte. Works fine is not how I would describe it. I think that was pretty much the only quoted text in my posting, and my question about how to create such a file was immediately below that paragraph, so I'm surprised that somebody would infer I was replying to something else. Track back up the thread and you'll see that the OP didn't mention an OS. True, but I wasn't replying to the OP. I was replying to a statement about how applications in the Unix world behave when presented with a filename containing a null byte. I thought it obvious that my question was regarding the unix world. He merely said that he was using zlib, and getting unfortunate results when he handled its output, so he could have been using any OS. Rhodri James assumed that the OS was Windows, but it wasn't until the 6th post that Steven D'Aprano mentioned Unix and null characters. And it was Steven D'Aprano's post to which I was replied with a question about how such a file could be created. I got sucked into the null trap - sorry - because I actually meant to generalise the discussion into ways of getting a range of unwanted characters into a file name and why its unwise to use a filename without checking it for characters the OS doesn't like before creating a file with it. I'm not disagreeing that in Unix you can create filenames with all sorts of inadvisable properties. 30 years ago I found backspaces and cursor control escape sequences to particularly amusing the first time I realized you could put them in filenames (that was under VMS, but you could do the same thing under Unix or most of the other OSes I've used). -- Grant -- http://mail.python.org/mailman/listinfo/python-list
[issue10076] Regex objects became uncopyable in 2.5
Senthil Kumaran orsent...@gmail.com added the comment: The patch looks reasonable to me. I was trying to find out why they got disabled after 2.5, but I don't see any reason. (There was an issue open and it was closed for no reason). So, I think, this should move forward, unless there is any technical objection to the patch. -- nosy: +orsenthil, pitrou stage: - patch review superseder: - MatchObjects not deepcopy()able versions: +Python 3.1 -Python 2.5, Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10076 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10111] Minor problems with PyFileObject documentation (Doc/c-api/file.rst)
New submission from Paul Bolle pebo...@tiscali.nl: 0) I ran into some (small) problems with the documentation added by revision 62195 (see issue 815646 for background). 1) A small patch that addresses two problems with the Python 2.7 documentation should be attached: - link three occurrences of GIL to the GIL entry in the glossary; and - add some example code to clarify the usage of PyFile_IncUseCount() andPyFile_DecUseCount(). 2) That patch also adds a link to the Thread State and the Global Interpreter Lock section to the GIL entry in the Documentation index. That is a separate problem. But fixing that minor problem could also be of help to people (like me) that need to better understand the GIL aspects of those two functions. -- assignee: d...@python components: Documentation files: python-2.7-Doc-gil.patch keywords: patch messages: 118742 nosy: d...@python, pebolle priority: normal severity: normal status: open title: Minor problems with PyFileObject documentation (Doc/c-api/file.rst) versions: Python 2.7 Added file: http://bugs.python.org/file19241/python-2.7-Doc-gil.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10111 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10100] fromfd is now available on all platforms
Amaury Forgeot d'Arc amaur...@gmail.com added the comment: You already changed the test in r84449! The doc still needs updating. -- nosy: +amaury.forgeotdarc ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10100 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue815646] thread unsafe file objects cause crash
Paul Bolle pebo...@tiscali.nl added the comment: please open a new issue and attach your patch(s) there Issue 10111 now tracks the documentation problems. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue815646 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10089] Add support for arbitrary -X options
Amaury Forgeot d'Arc amaur...@gmail.com added the comment: Why wouldn't they? A standard way to extend the command line options seems useful in all environments. Now the interpretation of these options are subject to variations of course... -- nosy: +amaury.forgeotdarc ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10089 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: [issue10089] Add support for arbitrary -X options
On Wed, Oct 13, 2010 at 07:01:40PM +, Antoine Pitrou wrote:
$ ./python -Xa,b=c,d -Xe,f=g=h -c import sys; print(sys.xoptions)
{'a': True, 'b': 'c', 'e': True, 'd': True, 'f': 'g=h'}
Docs should be updated too. I see that the values could either be True
or a string? Would this be perceived as a limitation?
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10098] intermittent failure in test_os
Amaury Forgeot d'Arc amaur...@gmail.com added the comment: _kill_with_event() does not wait for the subprocess to be ready. It seems to me that the following test is wrong: if m[0] == 0: It should be if m[0] == 1, since we want to check that the subprocess updated the shared memory. -- nosy: +amaury.forgeotdarc ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10098 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10089] Add support for arbitrary -X options
Amaury Forgeot d'Arc amaur...@gmail.com added the comment:
Well, the syntax allows to pass either a string value (because it's a substring
of the command line), or nothing.
When no value is passed, True seems better than None, because this allows the
usage of the get() method::
x = sys.xoptions.get('a')
or::
if sys.xoptions.get('a'):
this returns None if the option was not set, True if set to a empty value, and
a non-empty string if set to an actual value.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10089
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10110] Queue doesn't recognize it is full after shrinking maxsize
Changes by Ned Deily n...@acm.org: -- nosy: +rhettinger stage: - patch review versions: -Python 2.5, Python 2.6, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10110 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10019] json.dumps with indent = 0 not adding newlines
Florent Xicluna florent.xicl...@gmail.com added the comment: +1 to merge simplejson 2.1+ before 3.2 beta 1 (mid-november) -- nosy: +flox resolution: - accepted stage: - patch review type: - behavior versions: +Python 3.1, Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10019 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10099] socket.fromfd() documentation problem
Senthil Kumaran orsent...@gmail.com added the comment: I think, the original docs *is* pretty intuitive. It says Duplicate the file descriptor fd and build a socket object. No one will think that the this method will close the original fd. Person using this method might of course, explicitly close the original fd in some other part of the code (as he may use it after creating the socket object too). If at all anything is required, a line may be The original file descriptor is unaffected, but again that seems redundant to me. -- nosy: +orsenthil ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10099 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10100] fromfd is now available on all platforms
Senthil Kumaran orsent...@gmail.com added the comment: Docs changed committed in revision 85514. -- nosy: +orsenthil resolution: - fixed stage: needs patch - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10100 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10089] Add support for arbitrary -X options
Nick Coghlan ncogh...@gmail.com added the comment: I suggest using sys._xoptions to make it clearer that this is for CPython specific internal implementation runtime tweaks. We're putting it in sys so *we* can get at it, not applications. (It also makes it clear that other implementations aren't obliged to implement *any* particular interface to the -X options. Requiring that would go against the whole spirit of -X) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10089 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10106] missing packages
Senthil Kumaran orsent...@gmail.com added the comment: I would suggest to OP, to take it with python-help for the problem to be fixed. It's raised on python26 as well. Highly unlikely that anything is wrong with Python installation here. Marking it invalid and closing it. -- nosy: +orsenthil resolution: - invalid stage: - committed/rejected status: pending - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10106 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10099] socket.fromfd() documentation problem
Gergely Kálmán kalman.gerg...@duodecad.hu added the comment: You are perfectly right, the docs are pretty clear. Although fromfd means (or at least to me) to attach object to fd and not duplicate then attach to the duplicate. If someone forgets this particular behaviour and thinks that the function works as the name implies they're in trouble. Gergely Kalman -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10099 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10099] socket.fromfd() documentation problem
Senthil Kumaran orsent...@gmail.com added the comment: So, I assume that we just leave it as such and close the issue. I was thinking if anything needs to be updated for function __doc__ but even there 'the duplicate' word is explained. -- resolution: - invalid stage: - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10099 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10107] Quitting IDLE on Mac doesn't save unsaved code
Changes by Ronald Oussoren ronaldousso...@mac.com: -- nosy: +ronaldoussoren ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10107 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10079] idlelib for Python 3 with Guilherme Polo GSoC enhancements
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +kbk ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10079 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10112] Use -Wl, --dynamic-list=x.list, not -Xlinker -export-dynamic
New submission from Jan Kratochvil jan.kratoch...@redhat.com: FSF GDB (and future Fedora GDBs) became 250KB smaller than before. Python recently disabled this GDB build optimization so GDB is 250KB larger again. -rwxr-xr-x 1 4524488 gdb-7.1-19.fc13.x86_64/usr/bin/gdb -rwxr-xr-x 1 4266728 gdb-7.1-19dynamiclist.fc13.x86_64/usr/bin/gdb -rw-r--r-- 1 2364656 gdb-7.1-19.fc13.x86_64.rpm -rw-r--r-- 1 2274300 gdb-7.1-19dynamiclist.fc13.x86_64.rpm Some Python binaries/libraries could probably also benefit from smaller pageable code image. GDB regressed due to /usr/lib*/python*/config/Makefile contains: LINKFORSHARED=-Xlinker -export-dynamic and GDB thus has to use it even for its own link, effectively disabling its: -Wl,--dynamic-list=./proc-service.list AFAIK Python already contains the required exports list but it is used only on MS-Windows. It should be utilized even for GNU/Linux builds. -- components: Library (Lib) messages: 118756 nosy: dmalcolm, jankratochvil priority: normal severity: normal status: open title: Use -Wl,--dynamic-list=x.list, not -Xlinker -export-dynamic type: resource usage versions: Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10112 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10089] Add support for arbitrary -X options
Antoine Pitrou pit...@free.fr added the comment: (It also makes it clear that other implementations aren't obliged to implement *any* particular interface to the -X options. Requiring that would go against the whole spirit of -X) Agreed. I'll update the patch to use sys._xoptions. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10089 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10113] UnicodeDecodeError in mimetypes.guess_type on Windows
Changes by Vladimir Dmitriev vld...@gmail.com: -- components: Library (Lib), Windows files: mime content types.reg nosy: vldmit priority: normal severity: normal status: open title: UnicodeDecodeError in mimetypes.guess_type on Windows type: behavior versions: Python 2.7 Added file: http://bugs.python.org/file19242/mime content types.reg ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10113 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10113] UnicodeDecodeError in mimetypes.guess_type on Windows
New submission from Vladimir Dmitriev vld...@gmail.com:
Windows 7, Python 2.7
Some windows applications (QuickTime) add content-types to Windows registry
with non-ascii names. mimetypes in unaware of that and fails with
UnicodeDecodeError:
mimetypes.guess_type('test.js')
Traceback (most recent call last):
File stdin, line 1, in module
File c:\Python27\lib\mimetypes.py, line 294, in guess_type
init()
File c:\Python27\lib\mimetypes.py, line 355, in init
db.read_windows_registry()
File c:\Python27\lib\mimetypes.py, line 260, in read_windows_registry
for ctype in enum_types(mimedb):
File c:\Python27\lib\mimetypes.py, line 250, in enum_types
ctype = ctype.encode(default_encoding) # omit in 3.x!
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe0 in position 0: ordinal
not in range(128)
Example registry leaf is attached to previous message.
I believe the correct behavior would be either to wrap UnicodeDecodeError
exception and skip those content-typer or use .decode() method for registry
keys and get encoding using locale.getdefaultlocale()
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10113
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8340] bytearray undocumented on trunk
Changes by Paul Bolle pebo...@tiscali.nl: -- nosy: +pebolle ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8340 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10093] Warn when files are not explicitly closed
Amaury Forgeot d'Arc amaur...@gmail.com added the comment: Please add a similar warning in PC/_subprocess.c::sp_handle_dealloc() I just got caught by this in PyPy because some pipe handle relies on reference counting to be closed. This ad-hoc fix would suppress the warning: http://codespeak.net/pipermail/pypy-svn/2010-October/043746.html -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10093 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9980] str(float) failure
Kiriakos Vlahos pyscrip...@gmail.com added the comment: I would like to say that these are two separate issues. One is about the precision flag and the second about the exception masking flags of the FPU control words. Both issues affect a wider number of users than Python embedders using Delphi. For example C# uses 80 bit precision if I understand http://blogs.msdn.com/b/davidnotario/archive/2005/08/08/449092.aspx well. Also both issues are primarily documentation issues. If Python libraries make certain assumptions about the state of the FPU control word, these assumptions should be documented and users can then accommodate them. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9980 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10093] Warn when files are not explicitly closed
Antoine Pitrou pit...@free.fr added the comment: Please add a similar warning in PC/_subprocess.c::sp_handle_dealloc() I just got caught by this in PyPy because some pipe handle relies on reference counting to be closed. Do you want to provide a patch? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10093 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10114] compile() doesn't support the PEP 383 (surrogates)
New submission from STINNER Victor victor.stin...@haypocalc.com:
Example:
$ ./python
Python 3.2a3+ (py3k, Oct 15 2010, 14:31:59)
compile('', 'abc\uDC80', 'exec')
...
UnicodeEncodeError: 'utf-8' codec can't encode character '\udc80' in position
3: surrogates not allowed
Attached patch encodes manually the filename to utf-8 with surrogateescape.
I found this problem while testing Python with an ASCII locale encoding (LANG=C
./python Lib/test/regrtest.py). Example:
$ LANG=C ./python -m base64 -e setup.py
...
UnicodeEncodeError: 'utf-8' codec can't encode character '\udcc3' ...
--
components: Interpreter Core, Unicode
files: compile_surrogates.patch
keywords: patch
messages: 118762
nosy: haypo
priority: normal
severity: normal
status: open
title: compile() doesn't support the PEP 383 (surrogates)
versions: Python 3.2
Added file: http://bugs.python.org/file19243/compile_surrogates.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10114
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8293] HTTPSConnection.close() does not immediately close the connection.
R. David Murray rdmur...@bitdance.com added the comment: If I'm reading this thread correctly, this bug should be closed and a new one opened about SSL socket close. Antoine, does that sound correct? -- nosy: +r.david.murray ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8293 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8293] HTTPSConnection.close() does not immediately close the connection.
Changes by Giampaolo Rodola' g.rod...@gmail.com: -- nosy: +giampaolo.rodola ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8293 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10113] UnicodeDecodeError in mimetypes.guess_type on Windows
R. David Murray rdmur...@bitdance.com added the comment: This is a duplicate of #9291. -- nosy: +r.david.murray resolution: - duplicate stage: - committed/rejected status: open - closed superseder: - mimetypes initialization fails on Windows because of non-Latin characters in registry ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10113 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9291] mimetypes initialization fails on Windows because of non-Latin characters in registry
Changes by R. David Murray rdmur...@bitdance.com: -- nosy: +vldmit ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9291 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
