Re: Too many threads
In article <20100917052259.ga28...@cskk.homeip.net>, Cameron Simpson wrote: > On 16Sep2010 22:14, Ned Deily wrote: > | In article <20100917043826.ga21...@cskk.homeip.net>, > | Cameron Simpson wrote: > | > | > On 16Sep2010 09:55, mark.pellet...@asrcms.com > | > wrote: > | > | For some reason, the tasks I put into my thread pool occasionally get > | > | run more than once. > | > | > | > | Here's the code: > | > > | > You need to post your _exact_ code. I had to change: > | > from queue import Queue > | > into > | > from Queue import Queue > | > So: _do_ you have a "queue" (lowercase) module? Is it buggy? > | > | The OP is probably using Python 3. > > Ah, that explains the print(). Anyone know if print() is thread safe in > python 3? I vaguely recalled a similar problem and, on the second try, found it: http://bugs.python.org/issue6750 Looks like that fix should have been in Python 3.1.2. -- Ned Deily, n...@acm.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Too many threads
On 16Sep2010 22:14, Ned Deily wrote: | In article <20100917043826.ga21...@cskk.homeip.net>, | Cameron Simpson wrote: | | > On 16Sep2010 09:55, mark.pellet...@asrcms.com | > wrote: | > | For some reason, the tasks I put into my thread pool occasionally get | > | run more than once. | > | | > | Here's the code: | > | > You need to post your _exact_ code. I had to change: | > from queue import Queue | > into | > from Queue import Queue | > So: _do_ you have a "queue" (lowercase) module? Is it buggy? | | The OP is probably using Python 3. Ah, that explains the print(). Anyone know if print() is thread safe in python 3? -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ If I repent anything, it is very likely to be my good behavior. What demon possessed me that I behaved so well? - Henry David Thoreau -- http://mail.python.org/mailman/listinfo/python-list
Re: Too many threads
In article <20100917043826.ga21...@cskk.homeip.net>, Cameron Simpson wrote: > On 16Sep2010 09:55, mark.pellet...@asrcms.com > wrote: > | For some reason, the tasks I put into my thread pool occasionally get > | run more than once. > | > | Here's the code: > > You need to post your _exact_ code. I had to change: > from queue import Queue > into > from Queue import Queue > So: _do_ you have a "queue" (lowercase) module? Is it buggy? The OP is probably using Python 3. -- Ned Deily, n...@acm.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Too many threads
On 16Sep2010 09:55, mark.pellet...@asrcms.com wrote: | For some reason, the tasks I put into my thread pool occasionally get | run more than once. | | Here's the code: You need to post your _exact_ code. I had to change: from queue import Queue into from Queue import Queue So: _do_ you have a "queue" (lowercase) module? Is it buggy? Let's pretend not. Using the standard Queue module I can't reproduce your problem and your code looks correct to my eye. However, it is possible that the output stream used by print() is not thread safe. In which case your multiple outputs may stem from output buffer mishandling and not from your thread pool. Make another Queue to receice items, change _worker() to .put(item) onto it and collect form the queue and print in the program's main code. That way only one thread (the main one) will be trying to print(), removing the possibility of mismanagement of the output if print() is not thread safe. Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ Rules of Optimization: Rule 1: Don't do it. Rule 2 (for experts only): Don't do it yet. - M.A. Jackson -- http://mail.python.org/mailman/listinfo/python-list
Re: Very stupid question about a % symbol
On 9/16/2010 12:23 PM, J wrote:
Thanks for the replies... I KNEW there was a simple way to escape the
% but I had no idea what it was (I just had conviction).
I was thrown when the \ didn't escape it... never knew about %%. But
now I do! Thanks for the replies!
Doubling an escape char, whatever it is, is a common convention:
>>> print("Print a {{}} format string line this: {{{}}}".format(2))
Print a {} format string line this: {2}
--
Terry Jan Reedy
--
http://mail.python.org/mailman/listinfo/python-list
Re: The trouble with "dynamic attributes".
On 9/16/2010 5:46 PM, John Nagle wrote: By mistakenly posted this as a response to "compile Py2.6 on SL", you 1) confused the OP and 2) limited it audience -- I agree with your points, but would have missed this if I had had threads collapsed, as I usually do, since I have no interest in 2.6 compilation. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: os.path.normcase rationale?
On Thu, 16 Sep 2010 07:12:16 +1000, Ben Finney wrote: >> Yes, I know the behaviour is documented > > The docstring is fairly poor, IMO. You might want to submit a bug report > to improve it. The description in the library documentation is misleading: os.path.normcase(path) Normalize the case of a pathname. On Unix and Mac OS X, this returns the path unchanged; on case-insensitive filesystems, it converts the path to lowercase. On Windows, it also converts forward slashes to backward slashes. It implies that the behaviour depends upon the actual filesystem, which isn't the case. It only depends upon the platform, i.e. it assumes that all filenames are case-sensitive on Unix systems and case-insensitive on Windows. But Unix systems can access FAT/SMBFS/etc filesystems which are case-insensitive. -- http://mail.python.org/mailman/listinfo/python-list
Re: os.path.normcase rationale?
On Wed, 15 Sep 2010 14:49:09 +0100, Chris Withers wrote:
> I'm curious as to why, with a file called "Foo.txt"
> os.path.normcase('FoO.txt') will return "foo.txt" rather than "Foo.txt"?
normcase() doesn't look at the filesystem; it's just string manipulation.
--
http://mail.python.org/mailman/listinfo/python-list
Re: The trouble with "dynamic attributes".
On Fri, Sep 17, 2010 at 11:33 AM, moerchendiser2k3 wrote: > I am really sorry, but what are you talking about ? Hmmm, ...I have > problems to compile Python on SL, I did not ask anything about > "dynamic attribute". I don't get it... You are subscribed to the python mailing list. Check your subscription status with the link below. cheers James -- -- James Mills -- -- "Problems are solved by method" -- http://mail.python.org/mailman/listinfo/python-list
Re: WMI in Python
In message <257b2d99-03d4-491b-8f8b-dccd2bc10...@p22g2000pre.googlegroups.com>, alex23 wrote: > Lawrence D'Oliveiro wrote: > >> Why not just call Scriptomatic directly from within the Python script, >> then? > > Because Scriptomatic _generates scripts to access WMI_, that's what it > _does_. Precisely why I suggested running it on the fly. Because machine-generated code has no place in a source file to be maintained by a human. > Are you _seriously_ advocating writing Python code to fire up > a Windows application, programmatically manipulating a GUI to generate > more Python code for your original script to import or exec? If that’s the only way you can use Scriptomatic, then the stupidity lies with the one who suggested using such a brain-dead tool in the first place. > As you've made your disdain for Windows _perfectly_ clear through your > incessant sniping on this list, I think I'm pretty safe in saying you > have no idea of the quality of Scriptomatic's output. The quality of the output is not at issue here. The point is that it is machine-generated from an input command, therefore it makes sense from a maintenance viewpoint to keep the input command, not the machine-generated output, in the source file, and simply generate the latter from the former as part of the build process. Unfortunately, as you have made abundantly clear above, Scriptomatic is a GUI-only tool, totally unsuited to this sort of use. -- http://mail.python.org/mailman/listinfo/python-list
Re: The trouble with "dynamic attributes".
I am really sorry, but what are you talking about ? Hmmm, ...I have problems to compile Python on SL, I did not ask anything about "dynamic attribute". I don't get it... -- http://mail.python.org/mailman/listinfo/python-list
Re: File read from stdin and printed to temp file are not identicial?
On Fri, Sep 17, 2010 at 10:25 AM, Jean Luc Truchtersheim wrote: > Dear Fellow python users, > > Many thanks for your help. > > Those missing brackets were the cause of my problem. > > Now my program works as expected. > > Many, many heartfelt thanks. Glad to hear it! Do you understand why ? cheers James -- -- James Mills -- -- "Problems are solved by method" -- http://mail.python.org/mailman/listinfo/python-list
Re: File read from stdin and printed to temp file are not identicial?
Dear Fellow python users, Many thanks for your help. Those missing brackets were the cause of my problem. Now my program works as expected. Many, many heartfelt thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: data structure suggestion (native python datatypes or sqlite; compound select)
On 17/09/2010 00:56, Vlastimil Brom wrote:
2010/9/17 MRAB:
On 16/09/2010 23:11, Vlastimil Brom wrote:
...
I put together some code, which works as expected, but I suspect
somehow, that there must be better ways of doing it.
Two things I am not quite clear about are using the placeholders for
the data identifiers and "chaining" the SELECT parameters.
I Couldn't find a way to use "?" placeholder for table or column
names, hence I ended up using string interpolation for them and
placeholders for the data values, like.
curs.execute('SELECT * FROM "%s" WHERE "%s"==?' % (text_name,
index_col), (text_index,))
is there a better way or is it not supposed to supply these
identifiers programatically?
You would normally expect the structure of the database to be fixed and
only the contents to vary.
For getting the matching text indices given the tags, tag_values
combination I ended up with a clumsy query:
combined_query_list = ['SELECT "%s" FROM "%s" WHERE "%s"==?' %
(index_col, text_name, tag) for tag in tags]
sql_query = " INTERSECT ".join(combined_query_list)
curs.execute(sql_query, tag_values)
which produces e.g.:
SELECT "ind" FROM "n" WHERE "KC"==? INTERSECT SELECT "ind" FROM "n"
WHERE "VN"==?
or alternatively:
select_begin = 'SELECT "%s" FROM "%s" WHERE ' % (index_col, text_name)
where_subquery = " AND ".join('"%s"==?' % (tag,) for tag in tags)
sql_query = select_begin + where_subquery
with the resulting query string like:
SELECT "ind" FROM "n" WHERE "KC"==? AND "VN"==? ('12', '1')
(BTW, are these queries equivalent, as the outputs suggest, or are
there some distinctions to be aware of?)
Anyway, I can't really believe, this would be the expected way ...
If you're selecting rows of a table then using 'AND' would seem the
obvious way.
Thanks for the answer,
Well, that may be a part of the problem, the database structure is
going to be fixed once I'll have the text sources complete, but I was
trying to keep it more general, also allowing the identifiers to be
passed programmatically (based on the tagged text in question).
yes, I am just selecting rows - based on the combination of the column
values (which, I guess, might be an usual database approach(?).
However, I was unsure, whether it is usual to construct the query
string this way - partly using string interpolation or sequence
joining.
Or should there normally be no need for construct like the above and I
am doing something wrong in a more general sense?
In general you'd have a fixed database structure and fixed queries.
You'd design it so that you wouldn't have different columns for
different tagged texts, forcing you to change the database structure
when texts were added or removed, but no-one will come around to your
house to stop you creating queries dynamically. :-)
--
http://mail.python.org/mailman/listinfo/python-list
Re: File read from stdin and printed to temp file are not identicial?
On Fri, Sep 17, 2010 at 10:06 AM, MRAB wrote: > You're not closing f_in. That line should be: Although this _could_ be the problem (buffers not being flushed and the file being properly closed, etc) it could be something else... --James -- -- James Mills -- -- "Problems are solved by method" -- http://mail.python.org/mailman/listinfo/python-list
Re: File read from stdin and printed to temp file are not identicial?
On Fri, Sep 17, 2010 at 9:36 AM, Jean Luc Truchtersheim wrote: > Can anybody reproduce this behavior. Jean it would help if you could provide samples of your input and output files. I'm pretty sure I might have a clue as to what your problem might be, but I can't be sure until I see the input and resulting output files. cheers James -- -- James Mills -- -- "Problems are solved by method" -- http://mail.python.org/mailman/listinfo/python-list
Re: File read from stdin and printed to temp file are not identicial?
On 17/09/2010 00:36, Jean Luc Truchtersheim wrote: Hello, I am trying to read from stdin and dump what's read to a temporary file. My code works for small files but as soon as I have a file that has, e.g., more than 300 lines, there is always one and only one line that is truncated compared to the input. Here is my code: #- #! /usr/bin/env python import sys from tempfile import * if __name__ == "__main__": data = [] f_in = NamedTemporaryFile(suffix=".txt", delete=False) for line in sys.stdin: f_in.write(line) data.append(line) f_in.close f = open(f_in.name, 'rb') i=0 for line in f: if data[i] != line: print>>sys.stderr, "line %d:\nfile(%d):\"%s\"\narray(%d):\"%s\"" % (i+1, len(line), line, len(data[i]), data[i]) i += 1 sys.exit() #- I feel that I must be doing something very stupid, but I don't really know what. Any idea? Can anybody reproduce this behavior. Thanks a bunch for any help. Jean Luc. You're not closing f_in. That line should be: f_in.close() -- http://mail.python.org/mailman/listinfo/python-list
Re: File read from stdin and printed to temp file are not identicial?
On Thu, Sep 16, 2010 at 7:36 PM, Jean Luc Truchtersheim < jeanluc...@gmail.com> wrote: > Hello, > > I am trying to read from stdin and dump what's read to a temporary > file. My code works for small files but as soon as I have a file that > has, e.g., more than 300 lines, there is always one and only one line > that is truncated compared to the input. > > Here is my code: > > #- > #! /usr/bin/env python > > import sys > from tempfile import * > > if __name__ == "__main__": >data = [] >f_in = NamedTemporaryFile(suffix=".txt", delete=False) >for line in sys.stdin: >f_in.write(line) >data.append(line) >f_in.close > Does this need to be f_in.close() ? >f = open(f_in.name, 'rb') >i=0 >for line in f: >if data[i] != line: >print >>sys.stderr, "line > %d:\nfile(%d):\"%s\"\narray(%d):\"%s\"" % > (i+1, len(line), line, len(data[i]), data[i]) >i += 1 >sys.exit() > > #- > > I feel that I must be doing something very stupid, but I don't really > know what. > > Any idea? > > Can anybody reproduce this behavior. > > Thanks a bunch for any help. > > Jean Luc. > -- > http://mail.python.org/mailman/listinfo/python-list > -- Jason M. Swails Quantum Theory Project, University of Florida Ph.D. Graduate Student 352-392-4032 -- http://mail.python.org/mailman/listinfo/python-list
Re: data structure suggestion (native python datatypes or sqlite; compound select)
2010/9/17 MRAB :
> On 16/09/2010 23:11, Vlastimil Brom wrote:
>>
>>...
>> I put together some code, which works as expected, but I suspect
>> somehow, that there must be better ways of doing it.
>>
>> Two things I am not quite clear about are using the placeholders for
>> the data identifiers and "chaining" the SELECT parameters.
>>
>> I Couldn't find a way to use "?" placeholder for table or column
>> names, hence I ended up using string interpolation for them and
>> placeholders for the data values, like.
>> curs.execute('SELECT * FROM "%s" WHERE "%s"==?' % (text_name,
>> index_col), (text_index,))
>> is there a better way or is it not supposed to supply these
>> identifiers programatically?
>>
> You would normally expect the structure of the database to be fixed and
> only the contents to vary.
>
>> For getting the matching text indices given the tags, tag_values
>> combination I ended up with a clumsy query:
>>
>> combined_query_list = ['SELECT "%s" FROM "%s" WHERE "%s"==?' %
>> (index_col, text_name, tag) for tag in tags]
>> sql_query = " INTERSECT ".join(combined_query_list)
>> curs.execute(sql_query, tag_values)
>>
>> which produces e.g.:
>> SELECT "ind" FROM "n" WHERE "KC"==? INTERSECT SELECT "ind" FROM "n"
>> WHERE "VN"==?
>>
>> or alternatively:
>>
>> select_begin = 'SELECT "%s" FROM "%s" WHERE ' % (index_col, text_name)
>> where_subquery = " AND ".join('"%s"==?' % (tag,) for tag in tags)
>> sql_query = select_begin + where_subquery
>>
>> with the resulting query string like:
>> SELECT "ind" FROM "n" WHERE "KC"==? AND "VN"==? ('12', '1')
>>
>> (BTW, are these queries equivalent, as the outputs suggest, or are
>> there some distinctions to be aware of?)
>>
>> Anyway, I can't really believe, this would be the expected way ...
>>
> If you're selecting rows of a table then using 'AND' would seem the
> obvious way.
>
>
Thanks for the answer,
Well, that may be a part of the problem, the database structure is
going to be fixed once I'll have the text sources complete, but I was
trying to keep it more general, also allowing the identifiers to be
passed programmatically (based on the tagged text in question).
yes, I am just selecting rows - based on the combination of the column
values (which, I guess, might be an usual database approach(?).
However, I was unsure, whether it is usual to construct the query
string this way - partly using string interpolation or sequence
joining.
Or should there normally be no need for construct like the above and I
am doing something wrong in a more general sense?
Thanks again,
Vlastimil Brom
--
http://mail.python.org/mailman/listinfo/python-list
File read from stdin and printed to temp file are not identicial?
Hello, I am trying to read from stdin and dump what's read to a temporary file. My code works for small files but as soon as I have a file that has, e.g., more than 300 lines, there is always one and only one line that is truncated compared to the input. Here is my code: #- #! /usr/bin/env python import sys from tempfile import * if __name__ == "__main__": data = [] f_in = NamedTemporaryFile(suffix=".txt", delete=False) for line in sys.stdin: f_in.write(line) data.append(line) f_in.close f = open(f_in.name, 'rb') i=0 for line in f: if data[i] != line: print >>sys.stderr, "line %d:\nfile(%d):\"%s\"\narray(%d):\"%s\"" % (i+1, len(line), line, len(data[i]), data[i]) i += 1 sys.exit() #- I feel that I must be doing something very stupid, but I don't really know what. Any idea? Can anybody reproduce this behavior. Thanks a bunch for any help. Jean Luc. -- http://mail.python.org/mailman/listinfo/python-list
Re: The trouble with "dynamic attributes".
On Fri, Sep 17, 2010 at 9:27 AM, MRAB wrote: > For the work on updating the re module there was a discussion about > whether named capture groups should be available as attributes of the > match object or via subscripting (or both?). Subscripting seemed > preferable to me because: > > 1. Adding attributes looks too much like 'magic'. > > 2. What should happen if a group name conflicts with a normal attribute? > > 3. What should happen if a group name conflicts with a reserved word? > > For those reasons the new regex module uses subscripting. It's more > Pythonic, IMHO. I agree with 2) and 3) and in general this is probably a "good approach". cheers James -- -- James Mills -- -- "Problems are solved by method" -- http://mail.python.org/mailman/listinfo/python-list
Re: The trouble with "dynamic attributes".
On 16/09/2010 22:46, John Nagle wrote:
There's a tendency to use "dynamic attributes" in Python when
trying to encapsulate objects from other systems. It almost
works. But it's usually a headache in the end, and should be
discouraged. Here's why.
Some parsers, like BeautifulSoup, try to encapsulate HTML tag
fields as Python attributes. This gives trouble for several reasons.
First, the syntax for Python attributes and Python tags is different.
Some strings won't convert to attributes. You can crash BeautifulSoup
(which is supposed to be robust against bad HTML) by using a non-ASCII
character in a tag in an HTML document it is parsing.
Then there's the reserved word problem. "class" is a valid field
name in HTML, and a reserved word in Python. So there has to be a
workaround for reserved words.
There's also the problem that user-created attributes go into the
same namespace as other object attributes. This creates a vulnerability
comparable to MySQL injection. If an attacker controls the input
being parsed, they may be able to induce a store into something
they shouldn't be able to access.
This problem shows up again in "suds", the module for writing
SOAP RPC clients. This module tries to use attributes for
XML structures, and it almost works. It tends to founder when
the XML data model has strings that aren't valid attributes.
("-" appears frequently in XML fields, but is not valid in an
attribute name.)
Using a dictionary, or inheriting an object from "dict", doesn't
create these problems. The data items live in their own dictionary,
and can't clash with anything else. Of course, you have to write
tag['a']
instead of
tag.a
but then, at least you know what to do when you need
tag['class']
"suds", incidentally, tries to do both. They accept both
item.fieldname
and
item['fieldname']
But they are faking a dictionary, and it doesn't quite work right.
'fieldname' in item
works correctly, but the form to get None when the field is missing,
item.get('fieldname',None)
isn't implemented.
Much of the code that uses objects as dictionaries either predates
the days when you couldn't inherit from "dict", or was written by
Javascript programmers. (In Javascript, an object and a dictionary
are the same thing. In Python, they're not.) In new code, it's
better to inherit from "dict". It eliminates the special cases.
For the work on updating the re module there was a discussion about
whether named capture groups should be available as attributes of the
match object or via subscripting (or both?). Subscripting seemed
preferable to me because:
1. Adding attributes looks too much like 'magic'.
2. What should happen if a group name conflicts with a normal attribute?
3. What should happen if a group name conflicts with a reserved word?
For those reasons the new regex module uses subscripting. It's more
Pythonic, IMHO.
--
http://mail.python.org/mailman/listinfo/python-list
Re: data structure suggestion (native python datatypes or sqlite; compound select)
On 16/09/2010 23:11, Vlastimil Brom wrote:
Hi all,
I'd like to ask for suggestions regarding suitable datastracture for
storing textual metadata along with a plain text string.
The input format I have is simply text with custom tags like; I'd prefer to have this human readable format the original
data source.
For the application, the tags are extracted and stored in a custom
datastructure along with the plain text.The queries should either
return the tagset for a given text position (index) or reversely the
text indices for a given tag-value combination..
(I posted some more detailed remarks earlier, as I was beginning with this topic
http://mail.python.org/pipermail/python-list/2007-December/1130275.html
http://mail.python.org/pipermail/python-list/2008-May/1141958.html
Meanwhile I managed to get working code using python native
datastructures (nested defaultdicts and sets); however after some time
I am now planning to adapt this code for a web-based program and am
considering alternative datastructures, for simplicity I started with
sqlite (which should actually suffice for the data volume and traffic
in question).
I put together some code, which works as expected, but I suspect
somehow, that there must be better ways of doing it.
Two things I am not quite clear about are using the placeholders for
the data identifiers and "chaining" the SELECT parameters.
I Couldn't find a way to use "?" placeholder for table or column
names, hence I ended up using string interpolation for them and
placeholders for the data values, like.
curs.execute('SELECT * FROM "%s" WHERE "%s"==?' % (text_name,
index_col), (text_index,))
is there a better way or is it not supposed to supply these
identifiers programatically?
You would normally expect the structure of the database to be fixed and
only the contents to vary.
For getting the matching text indices given the tags, tag_values
combination I ended up with a clumsy query:
combined_query_list = ['SELECT "%s" FROM "%s" WHERE "%s"==?' %
(index_col, text_name, tag) for tag in tags]
sql_query = " INTERSECT ".join(combined_query_list)
curs.execute(sql_query, tag_values)
which produces e.g.:
SELECT "ind" FROM "n" WHERE "KC"==? INTERSECT SELECT "ind" FROM "n"
WHERE "VN"==?
or alternatively:
select_begin = 'SELECT "%s" FROM "%s" WHERE ' % (index_col, text_name)
where_subquery = " AND ".join('"%s"==?' % (tag,) for tag in tags)
sql_query = select_begin + where_subquery
with the resulting query string like:
SELECT "ind" FROM "n" WHERE "KC"==? AND "VN"==? ('12', '1')
(BTW, are these queries equivalent, as the outputs suggest, or are
there some distinctions to be aware of?)
Anyway, I can't really believe, this would be the expected way ...
If you're selecting rows of a table then using 'AND' would seem the
obvious way.
(I only marginally looked into sqlalchemy, which might simplify this a
bit, is this true? - Currently I only use the standard lib, depending
on the available server setup (python 2.6)).
Thanks in advance for any suggestions or pointers on both the sql
usage as well as the general datatype question.
regards,
Vlastimil Brom
--
http://mail.python.org/mailman/listinfo/python-list
Re: Deleting widgets from PyQt4 QFormWidget
On Wednesday 15 September 2010 18:53, Andrew wrote: > I'm trying to remove the widgets from the QFormLayout widget from > PyQt4. According to the documentation I should be able to use the > command .takeAt(int) which will delete the widget from the layout and > then return to me the QLayoutWidget. It will remove the widget from the layout but it won't delete it because it doesn't own it. The widget will still remain within its parent widget. You have to either reparent it to another widget, or delete it manually. [...] > Am I missing a step or is this just broken? I haven't been able to > find anything else on this issue yet. If it's broke, is there any > potential workaround? Try calling deleteLater() on each widget when you take it from the layout. Does that work? David -- http://mail.python.org/mailman/listinfo/python-list
data structure suggestion (native python datatypes or sqlite; compound select)
Hi all,
I'd like to ask for suggestions regarding suitable datastracture for
storing textual metadata along with a plain text string.
The input format I have is simply text with custom tags like ; I'd prefer to have this human readable format the original
data source.
For the application, the tags are extracted and stored in a custom
datastructure along with the plain text.The queries should either
return the tagset for a given text position (index) or reversely the
text indices for a given tag-value combination..
(I posted some more detailed remarks earlier, as I was beginning with this topic
http://mail.python.org/pipermail/python-list/2007-December/1130275.html
http://mail.python.org/pipermail/python-list/2008-May/1141958.html
Meanwhile I managed to get working code using python native
datastructures (nested defaultdicts and sets); however after some time
I am now planning to adapt this code for a web-based program and am
considering alternative datastructures, for simplicity I started with
sqlite (which should actually suffice for the data volume and traffic
in question).
I put together some code, which works as expected, but I suspect
somehow, that there must be better ways of doing it.
Two things I am not quite clear about are using the placeholders for
the data identifiers and "chaining" the SELECT parameters.
I Couldn't find a way to use "?" placeholder for table or column
names, hence I ended up using string interpolation for them and
placeholders for the data values, like.
curs.execute('SELECT * FROM "%s" WHERE "%s"==?' % (text_name,
index_col), (text_index,))
is there a better way or is it not supposed to supply these
identifiers programatically?
For getting the matching text indices given the tags, tag_values
combination I ended up with a clumsy query:
combined_query_list = ['SELECT "%s" FROM "%s" WHERE "%s"==?' %
(index_col, text_name, tag) for tag in tags]
sql_query = " INTERSECT ".join(combined_query_list)
curs.execute(sql_query, tag_values)
which produces e.g.:
SELECT "ind" FROM "n" WHERE "KC"==? INTERSECT SELECT "ind" FROM "n"
WHERE "VN"==?
or alternatively:
select_begin = 'SELECT "%s" FROM "%s" WHERE ' % (index_col, text_name)
where_subquery = " AND ".join('"%s"==?' % (tag,) for tag in tags)
sql_query = select_begin + where_subquery
with the resulting query string like:
SELECT "ind" FROM "n" WHERE "KC"==? AND "VN"==? ('12', '1')
(BTW, are these queries equivalent, as the outputs suggest, or are
there some distinctions to be aware of?)
Anyway, I can't really believe, this would be the expected way ...
(I only marginally looked into sqlalchemy, which might simplify this a
bit, is this true? - Currently I only use the standard lib, depending
on the available server setup (python 2.6)).
Thanks in advance for any suggestions or pointers on both the sql
usage as well as the general datatype question.
regards,
Vlastimil Brom
--
http://mail.python.org/mailman/listinfo/python-list
Re: The trouble with "dynamic attributes".
On Fri, Sep 17, 2010 at 7:46 AM, John Nagle wrote: > There's a tendency to use "dynamic attributes" in Python when > trying to encapsulate objects from other systems. It almost > works. But it's usually a headache in the end, and should be > discouraged. Here's why. What do you mean by "dynamic attributes " ? Can you show a simple code example ? cheers James -- -- James Mills -- -- "Problems are solved by method" -- http://mail.python.org/mailman/listinfo/python-list
The trouble with "dynamic attributes".
There's a tendency to use "dynamic attributes" in Python when
trying to encapsulate objects from other systems. It almost
works. But it's usually a headache in the end, and should be
discouraged. Here's why.
Some parsers, like BeautifulSoup, try to encapsulate HTML tag
fields as Python attributes. This gives trouble for several reasons.
First, the syntax for Python attributes and Python tags is different.
Some strings won't convert to attributes. You can crash BeautifulSoup
(which is supposed to be robust against bad HTML) by using a non-ASCII
character in a tag in an HTML document it is parsing.
Then there's the reserved word problem. "class" is a valid field
name in HTML, and a reserved word in Python. So there has to be a
workaround for reserved words.
There's also the problem that user-created attributes go into the
same namespace as other object attributes. This creates a vulnerability
comparable to MySQL injection. If an attacker controls the input
being parsed, they may be able to induce a store into something
they shouldn't be able to access.
This problem shows up again in "suds", the module for writing
SOAP RPC clients. This module tries to use attributes for
XML structures, and it almost works. It tends to founder when
the XML data model has strings that aren't valid attributes.
("-" appears frequently in XML fields, but is not valid in an
attribute name.)
Using a dictionary, or inheriting an object from "dict", doesn't
create these problems. The data items live in their own dictionary,
and can't clash with anything else. Of course, you have to write
tag['a']
instead of
tag.a
but then, at least you know what to do when you need
tag['class']
"suds", incidentally, tries to do both. They accept both
item.fieldname
and
item['fieldname']
But they are faking a dictionary, and it doesn't quite work right.
'fieldname' in item
works correctly, but the form to get None when the field is missing,
item.get('fieldname',None)
isn't implemented.
Much of the code that uses objects as dictionaries either predates
the days when you couldn't inherit from "dict", or was written by
Javascript programmers. (In Javascript, an object and a dictionary
are the same thing. In Python, they're not.) In new code, it's
better to inherit from "dict". It eliminates the special cases.
John Nagle
--
http://mail.python.org/mailman/listinfo/python-list
Re: socket.error: [Errno 98] Address already in use
> Where line 75 contains following: > s.bind((host, port)) As Tomas pointed out, you close conn, but you do not close the server socket 's' /Björn -- http://mail.python.org/mailman/listinfo/python-list
Re: Too much code - slicing
On Sep 16, 2:47 pm, Benjamin Kaplan wrote:
> On Thu, Sep 16, 2010 at 3:35 PM, DataSmash wrote:
> > I need to create a simple utility to remove characters from either the
> > right or left side of directories.
> > This works, but there has to be a better way. I tried to use a
> > variable inside the brackets but I can't get
> > that to work. Can anyone think of a way to do this with less code?
> > Thanks!
>
> > import os
>
> > dirs = filter(os.path.isdir, os.listdir(''))
> > for dir in dirs:
>
> > # Left side
>
>
>
> The int() type will convert a string to an int for you. So all you
> need to do is check the side and slice accordingly.
>
> if side=='l':
> code = dir[int(num):]
> else :
> code = dir[:-1*int(num)]
Much appreciated!
I thought I tried every combination, guess I didn't try this as it
works great.
Thanks again.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Too much code - slicing
On 09/16/2010 03:47 PM, Benjamin Kaplan wrote:
On Thu, Sep 16, 2010 at 3:35 PM, DataSmash wrote:
I need to create a simple utility to remove characters from either the
right or left side of directories.
This works, but there has to be a better way. I tried to use a
variable inside the brackets but I can't get
that to work. Can anyone think of a way to do this with less code?
Thanks!
import os
dirs = filter(os.path.isdir, os.listdir(''))
for dir in dirs:
# Left side
The int() type will convert a string to an int for you. So all you
need to do is check the side and slice accordingly.
if side=='l':
code = dir[int(num):]
else :
code = dir[:-1*int(num)]
I also like this construct that works, I think, since 2.6:
code = dir[int(num):] if side == 'l' else dir[:-1*int(num)]
-ak
--
http://mail.python.org/mailman/listinfo/python-list
Re: Too much code - slicing
On Thu, Sep 16, 2010 at 3:35 PM, DataSmash wrote:
> I need to create a simple utility to remove characters from either the
> right or left side of directories.
> This works, but there has to be a better way. I tried to use a
> variable inside the brackets but I can't get
> that to work. Can anyone think of a way to do this with less code?
> Thanks!
>
> import os
>
> dirs = filter(os.path.isdir, os.listdir(''))
> for dir in dirs:
>
> # Left side
The int() type will convert a string to an int for you. So all you
need to do is check the side and slice accordingly.
if side=='l':
code = dir[int(num):]
else :
code = dir[:-1*int(num)]
--
http://mail.python.org/mailman/listinfo/python-list
Too much code - slicing
I need to create a simple utility to remove characters from either the
right or left side of directories.
This works, but there has to be a better way. I tried to use a
variable inside the brackets but I can't get
that to work. Can anyone think of a way to do this with less code?
Thanks!
import os
dirs = filter(os.path.isdir, os.listdir(''))
for dir in dirs:
# Left side
if num == '1' and side == "l":
code = dir[1:]
if num == '2' and side == "l":
code = dir[2:]
if num == '3' and side == "l":
code = dir[3:]
if num == '4' and side == "l":
code = dir[4:]
if num == '5' and side == "l":
code = dir[5:]
if num == '6' and side == "l":
code = dir[6:]
if num == '7' and side == "l":
code = dir[7:]
if num == '8' and side == "l":
code = dir[8:]
if num == '9' and side == "l":
code = dir[9:]
if num == '10' and side == "l":
code = dir[10:]
# Right side
if num == '1' and side == "r":
code = dir[:-1]
if num == '2' and side == "r":
code = dir[:-2]
if num == '3' and side == "r":
code = dir[:-3]
if num == '4' and side == "r":
code = dir[:-4]
if num == '5' and side == "r":
code = dir[:-5]
if num == '6' and side == "r":
code = dir[:-6]
if num == '7' and side == "r":
code = dir[:-7]
if num == '8' and side == "r":
code = dir[:-8]
if num == '9' and side == "r":
code = dir[:-9]
if num == '10' and side == "r":
code = dir[:-10]
print " Renaming "+dir+" to "+code
os.rename(dir, code)
--
http://mail.python.org/mailman/listinfo/python-list
Re: Problem building python 2.7 with --enable-shared
I digged into this one step further: I compared the output of configure and make. For configure, there is no change. But for make, I found out something, that I didn't expected: When configure is called with a prefix to a location, where a valid python installation already exists, it uses these locations for gcc, although it shouldn't (we are about to compile a clean, brandnew python version, so why should it include parts of an existing python installation?). Additionally to the default include paths gcc [...] -I. -IInclude -I./Include -I/usr/local/include -I/usr/src/redhat/BUILD/Python-2.7/Include -I/usr/src/redhat/BUILD/Python-2.7 [...] the path of the existing Python installation is added: -I/test/python2/include And additionally the default library paths and libs gcc [...] -L/usr/local/lib -L. -lpython2.7 [...] the path of the existing Python libs is added: gcc [...9 -L/vrmd/python2/lib So linking to python2.7 during make doesn't link to the freshly build library, but to the existing in the system. And at least in my case, where I'm switching from a python installation without --enabled-shared to a new version including --enable-shared, this causes serious problems: /usr/bin/ld: /vrmd/python2/lib/libpython2.7.a(abstract.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC -/vrmd/python2/lib/libpython2.7.a: could not read symbols: Bad value -collect2: ld returned 1 exit status Is there any option I missed in configure or any variable that I might set to correct this? Or should a file a bug to the python developers? Deinstalling python before compiling or renaming the python directory is not a good solution. Kind regards Marten -- http://mail.python.org/mailman/listinfo/python-list
Re: Problem building python 2.7 with --enable-shared
The strange thing is: This only happens when I'm compiling through rpmbuild. Issuing the same commands (configure --enable-shared, make, make install) directly on the shell works fine... -- http://mail.python.org/mailman/listinfo/python-list
Re: Problem building python 2.7 with --enable-shared
On Thursday 16 September 2010, it occurred to Marten Lehmann to exclaim: > Hello, > > I've build python 2.7 successfully by just calling configure, make and > make install. But to use python within PostgreSQL, I need to built > python with --enable-shared. > > I tried to do so (configure --enable-shared), but I see these lines > several times in the log (but compiling continues): Did you start with a clean source tree? If you just reconfigured and called make again in the source tree where you already had built Python, make will just re-use the old object files, which would be built without -fPIC and whatever other options are required for the shared library linking to work. > > /usr/bin/ld: /test/python/lib/libpython2.7.a(abstract.o): relocation > R_X86_64_32 against `a local symbol' can not be used when making a > shared object; recompile with -fPIC > /test/python/lib/libpython2.7.a: could not read symbols: Bad value > collect2: ld returned 1 exit status If you get this when building from a fresh source tree, I can't explain why this might happen, but, again, starting afresh, this time adding -fPIC to the CFLAGS, may work. But I expect that the configure script, or makefile, or libtool, or whatever, should do this, hence my guess that maybe you're using leftover object files. (hint: make clean) > > Later, the compiling process ends completely: > > PYTHONPATH=/var/tmp/vrmd-python2-root/test/python/lib/python2.7 > LD_LIBRARY_PATH=/usr/src/redhat/BUILD/Python-2.7: \ > ./python -Wi -tt > /var/tmp/vrmd-python2-root/test/python/lib/python2.7/compileall.py \ > -d /test/python/lib/python2.7 -f \ > -x > 'bad_coding|badsyntax|site-packages|lib2to3/tests/data' \ > /var/tmp/vrmd-python2-root/test/python/lib/python2.7 > Traceback (most recent call last): >File > "/var/tmp/vrmd-python2-root/test/python/lib/python2.7/compileall.py", > line 17, in > import struct >File > "/var/tmp/vrmd-python2-root/test/python/lib/python2.7/struct.py", line > 1, in > from _struct import * > ImportError: No module named _struct > make: *** [libinstall] Error 1 > > Any ideas, what I have done wrong? > > Kind regards > Marten -- http://mail.python.org/mailman/listinfo/python-list
Re: program organization question for web development with python
Another pretty web framework is Bottle (http://bottle.paws.de). It is very easy to setup and use. I use it to tasks like this one you want. []s iuri On Thu, Sep 16, 2010 at 3:01 PM, MRAB wrote: > On 16/09/2010 18:19, Hans wrote: > >> On Sep 15, 5:33 pm, James Mills wrote: >> >>> On Thu, Sep 16, 2010 at 10:14 AM, Hans wrote: >>> I'm new to this area. Please allow me to ask some (maybe stupid) questions. >>> >>> Without reading the rest of your post too much. Designs are up to you, >>> I can't comment. >>> >>> I can only share in a fairly common view, and that is, we'd encourage >>> you to use a web framework >>> as opposed to plain old CGI. >>> >>> cheers >>> james >>> >>> -- >>> -- James Mills >>> -- >>> -- "Problems are solved by method" >>> >> >> Hi James, >> >> Thanks for response. >> Maybe I did not make my question clear. I never tried python web >> programing before, so I want to start from CGI. >> >> I read something about web framework like django, but seems it's a >> little bit complicated. my task is actually very simple: get search >> string from input, and then search database, print search result. I >> thought CGI should be good enough to do this. >> >> I don't have any idea about how to organize those cgi codes, so what >> I'm asking is: >> >> 1. do I have to have each single file for each hyper-link? Can I put >> them together? how? >> 2. how can I pass a db_cursor to another file? can I use db_cursor as >> a parameter? >> >> I recently wrote a web-based program using CherryPy, which was very > straightforward. That might suit your needs. > > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://mail.python.org/mailman/listinfo/python-list
Re: program organization question for web development with python
On 16/09/2010 18:19, Hans wrote: On Sep 15, 5:33 pm, James Mills wrote: On Thu, Sep 16, 2010 at 10:14 AM, Hans wrote: I'm new to this area. Please allow me to ask some (maybe stupid) questions. Without reading the rest of your post too much. Designs are up to you, I can't comment. I can only share in a fairly common view, and that is, we'd encourage you to use a web framework as opposed to plain old CGI. cheers james -- -- James Mills -- -- "Problems are solved by method" Hi James, Thanks for response. Maybe I did not make my question clear. I never tried python web programing before, so I want to start from CGI. I read something about web framework like django, but seems it's a little bit complicated. my task is actually very simple: get search string from input, and then search database, print search result. I thought CGI should be good enough to do this. I don't have any idea about how to organize those cgi codes, so what I'm asking is: 1. do I have to have each single file for each hyper-link? Can I put them together? how? 2. how can I pass a db_cursor to another file? can I use db_cursor as a parameter? I recently wrote a web-based program using CherryPy, which was very straightforward. That might suit your needs. -- http://mail.python.org/mailman/listinfo/python-list
Problem building python 2.7 with --enable-shared
Hello, I've build python 2.7 successfully by just calling configure, make and make install. But to use python within PostgreSQL, I need to built python with --enable-shared. I tried to do so (configure --enable-shared), but I see these lines several times in the log (but compiling continues): /usr/bin/ld: /test/python/lib/libpython2.7.a(abstract.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC /test/python/lib/libpython2.7.a: could not read symbols: Bad value collect2: ld returned 1 exit status Later, the compiling process ends completely: PYTHONPATH=/var/tmp/vrmd-python2-root/test/python/lib/python2.7 LD_LIBRARY_PATH=/usr/src/redhat/BUILD/Python-2.7: \ ./python -Wi -tt /var/tmp/vrmd-python2-root/test/python/lib/python2.7/compileall.py \ -d /test/python/lib/python2.7 -f \ -x 'bad_coding|badsyntax|site-packages|lib2to3/tests/data' \ /var/tmp/vrmd-python2-root/test/python/lib/python2.7 Traceback (most recent call last): File "/var/tmp/vrmd-python2-root/test/python/lib/python2.7/compileall.py", line 17, in import struct File "/var/tmp/vrmd-python2-root/test/python/lib/python2.7/struct.py", line 1, in from _struct import * ImportError: No module named _struct make: *** [libinstall] Error 1 Any ideas, what I have done wrong? Kind regards Marten -- http://mail.python.org/mailman/listinfo/python-list
Re: program organization question for web development with python
On Sep 15, 5:33 pm, James Mills wrote: > On Thu, Sep 16, 2010 at 10:14 AM, Hans wrote: > > I'm new to this area. Please allow me to ask some (maybe stupid) > > questions. > > Without reading the rest of your post too much. Designs are up to you, > I can't comment. > > I can only share in a fairly common view, and that is, we'd encourage > you to use a web framework > as opposed to plain old CGI. > > cheers > james > > -- > -- James Mills > -- > -- "Problems are solved by method" Hi James, Thanks for response. Maybe I did not make my question clear. I never tried python web programing before, so I want to start from CGI. I read something about web framework like django, but seems it's a little bit complicated. my task is actually very simple: get search string from input, and then search database, print search result. I thought CGI should be good enough to do this. I don't have any idea about how to organize those cgi codes, so what I'm asking is: 1. do I have to have each single file for each hyper-link? Can I put them together? how? 2. how can I pass a db_cursor to another file? can I use db_cursor as a parameter? Thanks again! Hans -- http://mail.python.org/mailman/listinfo/python-list
Too many threads
For some reason, the tasks I put into my thread pool occasionally get run more than once. Here's the code: # --- from threading import Thread from queue import Queue import subprocess class ThreadPool(object): def __init__(self, thread_count): ''' Argument thread_count is the maximum number of theads. ''' self.thread_count = thread_count self.queue = Queue() # create and start the threads for i in range(self.thread_count): t = Thread(target=self._worker) t.daemon = True t.start() def _worker(self): ''' A "private" method that pulls tasks off the queue and does something with them. ''' while True: item = self.queue.get() print(item) self.queue.task_done() def run(self, tasklist): ''' Put tasks in the queue. ''' for item in tasklist: self.queue.put(item) # block until all threads are done. self.queue.join() # - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - tasklist = [ 'task1', 'task2', 'task3', 'task4', 'task5', 'task6', 'task7', 'task8', 'task9', 'task10', 'task11', 'task12', 'task13', 'task14', 'task15', 'task16'] if __name__ == '__main__': t = ThreadPool(3) t.run(tasklist) #--- And here's some typical output: task1 task2 task2 task3 task4 task5 task5 task6 task7 task8 task8 task9 task8 task9 task10 task11 task11 task12 task13 task13 task14 task15 task15 task16 I only want a task to get fired off once. What am I doing wrong? Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: socket.error: [Errno 98] Address already in use
On Sep 15, 11:58 am, Grant Edwards wrote: > On 2010-09-15, cerr wrote: > > > > > > > I get a socket error "[Errno 98] Address already in use" when i try to > > open a socket that got closed before with close(). How come close() > > doesn't close the socket properly? > > My socket code : > > > s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) > > s.bind((host, port)) > > s.listen(1) > > ... > > ... > > ... > > while loop: > > conn, addr = s.accept() > > while conn and loop: > > ... > > ... > > ... > > conn.close() > > At what line does the error occur? The whole message I get looks like: Traceback (most recent call last): File "./checkGPIO.py", line 148, in main() File "./checkGPIO.py", line 75, in main s.bind((host, port)) File "", line 1, in bind socket.error: [Errno 98] Address already in use Where line 75 contains following: s.bind((host, port)) > > To what does the phrase "open a socket" refer? create a listening socket...? > Have you tried the usual solution of setting the SO_REUSEADDR option > on the socket before calling bind? yep, that did it for me, thanks a lot! :) > http://www.google.com/search?q=socket+%27address+already+in+use%27 Google's your friend if you can read ;) > > -- > Grant Edwards grant.b.edwards Yow! I own seven-eighths of > at all the artists in downtown > gmail.com Burbank! -- http://mail.python.org/mailman/listinfo/python-list
Re: socket.error: [Errno 98] Address already in use
On Sep 15, 5:51 pm, "jipalaciosort...@gmail.com" wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > El 15/09/2010 20:58, Grant Edwards escribió: > > > > > On 2010-09-15, cerr wrote: > > >> I get a socket error "[Errno 98] Address already in use" when i > >> try to open a socket that got closed before with close(). How > >> come close() doesn't close the socket properly? My socket code : > > >> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) > >> s.bind((host, port)) s.listen(1) ... ... ... while loop: conn, > >> addr = s.accept() while conn and loop: ... ... ... conn.close() > > > At what line does the error occur? > > > To what does the phrase "open a socket" refer? > > > Have you tried the usual solution of setting the SO_REUSEADDR > > option on the socket before calling bind? > > >http://www.google.com/search?q=socket+%27address+already+in+use%27 > > Maybe, you have any other proccess in your system using your listen > port, for example apache... Nope negative, I have one process on my system only that occupies port 1514. > > - -- > _ _ _ _ _ _ _ _ _ _ _ > Jose Ignacio Palacios Ortega /_ _/ / / / _ / / _ / > Telf: +34 637 058 813 / / / / / /_ / / / / / / > Correo-e: jipalaciosort...@gmail.com _ / / / / / _ _ _/ / / / / > Msn: jipalaciosort...@gmail.com / /_ / / / / / / / /_/ / > ID firma PGP: 0x0EB87E48 \ _ _ / /_/ /_/ /_ _ _/ > Huella PGP:61CC 5DA0 827B C3AB F83C 2A55 78AF B317 0EB8 7E48 > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.4.10 (MingW32) > Comment: Using GnuPG with Mozilla -http://enigmail.mozdev.org/ > > iEYEARECAAYFAkyRafgACgkQeK+zFw64fkjH2wCffe4v8ho2z4d8LWaPaiJRu0OZ > 4cgAniOoR70hu7UylkpgAr3JI5hxNXYP > =MoYK > -END PGP SIGNATURE- -- http://mail.python.org/mailman/listinfo/python-list
Python 2.7 Won't Build
I am trying to rebujild the 2.7 maintenance branch and get this error on Ubuntu 10.04.1 LTS: XXX lineno: 743, opcode: 0 Traceback (most recent call last): File "/usr/local/src/python-2.7-maint-svn/Lib/site.py", line 62, in import os File "/usr/local/src/python-2.7-maint-svn/Lib/os.py", line 743, in def urandom(n): SystemError: unknown opcode I installed it successfully once so I may be getting conflicts, but I can't figure it out. There were some similar bugs reported in previous versions but I didn't see a clear solution. I have unset my PYTHONPATH and LD_LIBRARY_PATH, but python2.7 is my default python. I guess my next step will be to manually remove python 2.7 unless I hear some solutions soon. Thanks, -Tom Thomas M. Browder, Jr. Niceville, Florida USA -- http://mail.python.org/mailman/listinfo/python-list
Re: Very stupid question about a % symbol
Ha, I had this same problem, but I was trying to do dynamic formatting:
("%%%s" % format) % number
where "format" is a python-ized fortran format string (i.e. "9.4E"). Looks
kinda weird and less elegant than the {0:{1}}-type .format() syntax, but at
least it preserves backwards compatibility to pythons older than 2.6.
Before I found out how to cancel the %, the statement looked like ("%"+"%s"
% format) % number. Not much of a change, but still a nice thing to know
since I play with %s a lot.
Thanks! (even though I'm not the original asker)
Jason
On Thu, Sep 16, 2010 at 12:09 PM, Grant Edwards wrote:
> On 2010-09-16, J wrote:
>
> > Reported memory amounts are within 10% tolerance
>
> >>> "Reported memory amounts are within %d%% tolerance" % 10
> 'Reported memory amounts are within 10% tolerance'
>
> --
> Grant Edwards grant.b.edwardsYow! It's the RINSE
> CYCLE!!
> at They've ALL IGNORED the
> gmail.comRINSE CYCLE!!
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
Jason M. Swails
Quantum Theory Project,
University of Florida
Ph.D. Graduate Student
352-392-4032
--
http://mail.python.org/mailman/listinfo/python-list
Re: Very stupid question about a % symbol
On Thu, Sep 16, 2010 at 12:09, Grant Edwards wrote: > On 2010-09-16, J wrote: > >> Reported memory amounts are within 10% tolerance > "Reported memory amounts are within %d%% tolerance" % 10 > 'Reported memory amounts are within 10% tolerance' Thanks for the replies... I KNEW there was a simple way to escape the % but I had no idea what it was (I just had conviction). I was thrown when the \ didn't escape it... never knew about %%. But now I do! Thanks for the replies! -- http://mail.python.org/mailman/listinfo/python-list
Re: Very stupid question about a % symbol
On 09/16/10 10:25, J wrote: OK, this is a very stupid question about a very simple topic, but print "Reported memory amounts are within %s%s tolerance" % (self.mem_tolerance,'%') Is there a better way to print a '%' in the string when also using formating? I've tried things like this: print "blahblahblah %s \%" % variable So close print "blah %s %%" % variable -tkc -- http://mail.python.org/mailman/listinfo/python-list
Re: Very stupid question about a % symbol
On 2010-09-16, J wrote: > Reported memory amounts are within 10% tolerance >>> "Reported memory amounts are within %d%% tolerance" % 10 'Reported memory amounts are within 10% tolerance' -- Grant Edwards grant.b.edwardsYow! It's the RINSE CYCLE!! at They've ALL IGNORED the gmail.comRINSE CYCLE!! -- http://mail.python.org/mailman/listinfo/python-list
Re: Very stupid question about a % symbol
On 17 September 2010 01:25, J wrote: > Is there a better way to print a '%' in the string when also using > formating? > I believe %% will escape the % and prints it straight out. Cheers, Xav -- http://mail.python.org/mailman/listinfo/python-list
Very stupid question about a % symbol
OK, this is a very stupid question about a very simple topic, but Google is failing me this morning... I'm trying to print a string that looks like this: Reported memory amounts are within 10% tolerance and the print line looks (for now) like this: print "Reported memory amounts are within %s%s tolerance" % (self.mem_tolerance,'%') Is there a better way to print a '%' in the string when also using formating? That works, but I've been looking for a way that looks... less kludgey. I've tried things like this: print "blahblahblah %s \%" % variable but the first %s forces every other % to be interpreted as additional format markers. So while what I'm doing works, I was hoping someone could either confirm that that's the right way, or show me a better way. Cheers Jeff -- http://mail.python.org/mailman/listinfo/python-list
compile Py2.6 on SL
Hi, I have some trouble with Python on Snow Leopard (10.6.3). I compile Python as a framework(for 32/64bit) without any problems. But implementing the lib in my C app, I get the following error on linking: Undefined symbols: "_Py_InitModule4_64", referenced from: RegisterModule_BPY(char const*, PyMethodDef*, char const*, _object*, int)in i_moduleobject.o ld: symbol(s) not found I read a lot of stuff about this problem, but nothing helped me. My app is in 64-bit and I compiled Python for 32/64bit. When I compile exactly the same package with the same configure flags under Leopard (10.5.8) it works fine. Any ideas? Thanks a lot! Bye, moerchendiser2k3 -- http://mail.python.org/mailman/listinfo/python-list
Stream Audio to Web Clients
I'm looking for an example (perhaps with red5) for this scenario: 1. I install some server tools on my ubuntu box 2. I place an mp3 and an m4a in a special location 3. I click a play button on a web page and both songs play in random order AJ ONeal -- http://mail.python.org/mailman/listinfo/python-list
Re: WMI in Python
On Sep 16, 12:39 pm, alex23 wrote: > Lawrence D'Oliveiro wrote: > > Why not just call Scriptomatic directly from within the Python script, then? > > Because Scriptomatic _generates scripts to access WMI_, that's what it > _does_. Are you _seriously_ advocating writing Python code to fire up > a Windows application, programmatically manipulating a GUI to generate > more Python code for your original script to import or exec? > > For your question to make any real sense, you'd want to ask "Why not > access WMI directly...", which you might notice was the first > recommendation I made. > > > Machine-generated code has no place in a source file to be maintained by a > > human. > > As you've made your disdain for Windows _perfectly_ clear through your > incessant sniping on this list, I think I'm pretty safe in saying you > have no idea of the quality of Scriptomatic's output. I don't > understand what possibly makes you believe you're qualified to make > such a statement. Thank you all looks I got lot of reading to be done now based on all your suggestions. -- http://mail.python.org/mailman/listinfo/python-list
Re: Multiprocessing Queue strange behavior
Hi, thanks for the answer.
I thought about that, but the problem is that I found the problem in code
that *was* using the Queue between processes. This code for example fails
around 60% of the time in one of our linux machines (raising an Empty
exception):
from processing import Queue, Process
import time
def Consume(queue):
print queue.get_nowait()
if __name__ == '__main__':
queue = Queue()
queue.put('x')
p = Process(target=Consume, args=(queue,))
p.start()
p.join()
Again, putting a sleep call before starting the consumer process makes the
code work all time. That is the main reason I believe there is a problem in
the Queue code somewhere.
Cheers,
On Wed, Sep 15, 2010 at 5:47 PM, MRAB wrote:
> On 15/09/2010 21:10, Bruno Oliveira wrote:
>
>> Hi list,
>>
>> I recently found a bug in my company's code because of a strange
>> behavior using multiprocessing.Queue. The following code snippet:
>>
>> from multiprocessing import Queue
>>
>> queue = Queue()
>> queue.put('x')
>> print queue.get_nowait()
>> Fails with:
>>
>> ...
>> File
>>
>> "E:\Shared\dist-0902\i686.win32\processing-0.52\lib\site-packages\processing\queue.py",
>> line 153, in getNoWait
>> return self.get(False)
>> File
>>
>> "E:\Shared\dist-0902\i686.win32\processing-0.52\lib\site-packages\processing\queue.py",
>> line 129, in get
>> raise Empty
>> Queue.Empty
>>
>> Strangely, changing this to:
>>
>> queue = Queue()
>> queue.put('x')
>> time.sleep(0.1) # <<<
>> print queue.get_nowait()
>> Works as expected. Using the original snippet changing the import to
>> threading's Queue also works.
>>
>> It seems like there's a bug in multiprocessing's Queue implementation.
>> Opinions?
>>
>> I don't think it's a bug as such.
>
> The purpose of the multiprocessing queue is to transfer data between
> different processes, which don't have a shared address space (unlike
> threads, which do).
>
> The transfer involves passing the data between the processes via a
> pipe. This is done in a background thread and takes some time to
> complete, so the data won't appear immediately. It looks like it
> doesn't matter that the putter and the getter happen to be in the same
> process, possibly because no-one expected that someone would use a
> multiprocessing queue within the same process like that, so it doesn't
> check for a shortcut.
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python autocomplete module
On Thu, Sep 16, 2010 at 5:34 PM, dusans wrote: > Is there a python module to make autocomplete suggestion easy. > Try 'rlcompleter' module. Though I haven't tried it myself, just used it in .pythonrc for auto-completion in intepretor mode. http://docs.python.org/library/rlcompleter.html -- ~l0nwlf -- http://mail.python.org/mailman/listinfo/python-list
Python autocomplete module
Is there a python module to make autocomplete suggestion easy. I would build an api reference as a string: CATALOG1.SCHEME1.TABLE1.column1 CATALOG1.SCHEME1.TABLE1.column2 CATALOG1.SCHEME1.TABLE1.column3 . . CATALOG10.SCHEME10.TABLE100.column1 CATALOG10.SCHEME10.TABLE100.column2 CATALOG10.SCHEME10.TABLE100.column3 And when the user types and want suggestions he would get a nice list. Example: I trype "CATALOG10.SCHEME10." then i request autocomplete and it would return the tables in the specified scheme. Tnx -- http://mail.python.org/mailman/listinfo/python-list
Re: Accessing windoze file attributes
@Dave Grateful thanks for your web site suggestions, and I will look them up. @Lawrence Pleasee accept my apologies. I mistook your brevity for a superior snotty attitude. -- http://mail.python.org/mailman/listinfo/python-list
Re: accessing a text file
On Sep 9, 10:09 pm, Nobody wrote: > On Wed, 08 Sep 2010 03:30:00 -0700, Baba wrote: > > Who is licensed to judge what can and cannot be posted as a question? > > Exactly the same set of people who are licensed to judge what can and > cannot be posted as an answer. > > If you don't like the responses you get here, you could try posting your > questions on 4chan. If nothing else, that will give you a whole new > perspective on what an "unfriendly" response really looks like. I would to apologise to anyone who might have been upset or offended by my reaction. I am new to forums (as i am new to programming) so i hope i might be forgiven for not appreciating the true value of the support one can receive on this forum. I wish to reiterate that i underatand that experts provide help free of charge and in their spare time so there's no point for me to like or dislike the style of an answer. Baba -- http://mail.python.org/mailman/listinfo/python-list
Re: business date and calendar libraries?
On 13/09/2010 18:06, Chris Withers wrote: Hi All, I'm wondering what libraries people would use to answer the following questions relating to business days: - on a naive level; "what's give me the last business day" (ie: skipping weekends) - on a less-naive level; same question but taking into account public holidays - on a horrific level; same question, but taking into account business days of a particular market (NYSE, LSE, etc) Hi Chris, Having written one of these for UK holidays, it was easy once I obtained a routine to calculate the date of Easter :) We decided that editing a list of holidays each year was "too hard" and so they would always be calculated so.. I had a "isWorkday()" routine that returned false for everything below, and true otherwise: Every Sat and Sun Jan 1st, and the first Monday in Jan The day after Easter day The first Monday in May (Month = May and day <= 7 and dow = Monday) The last Monday in May (day >= 25 The last Monday in August (day >= 25 Every day from 23rd Dec to the end of the year. (unique to company). I believe that officially, Christmas holiday is the first week-day on or after 25th Dec. Boxing Day/St Stephens holiday is the week-day after the Christmas holiday date above. Hope this helps. Ian -- http://mail.python.org/mailman/listinfo/python-list
Re: Accessing windoze file attributes
On 2:59 PM, Douglas wrote: @Diez and @David Thanks guys. Both excellent leads. Colour me happy. I can now make progress and meet some deadlines. :) @Lawrence Why reinvent rsync? In what way is rsync relevant to the stated problem? Did you actually READ the question? Note: I use Linux at home (yes, even rsync) and very much prefer it to Windows. However, at work I have no choice but to use the resources the company provides - which is Windows; have you got that straight now? If you have something more helpful and mature to say, relevant to the actual question, then I will gladly hear it, and even apologise. Otherwise, don't bother. I already enough from respondents who were far more adult, knowledgeable and helpful ... and all the more impressive in comparison. Sincerely, -- Douglas Let's see. You're a first-time poster on this forum, and you choose to attack Lawrence, who was giving you a succinct reference to a possible solution to your problem. So how does mentioning a program that could directly solve your problem constitute immature and unhelpful response? Have you checked out any of the several rsync implementations on Windows? Some require cygwin, but there are a couple that would seem not to. Maybe try http://www.aboutmyip.com/AboutMyXApp/DeltaCopy.jsp or http://www.itefix.no/i2/node/10650 There are several reasons why any particular one of these may not be appropriate for your needs, but being stuck on Windows isn't really one of them. DaveA -- http://mail.python.org/mailman/listinfo/python-list
MAKE UPTO $5000 P/M $2000 IN FIRST 30 DAYS! NO INVESTMENT
MAKE UPTO $5000 P/M $2000 IN FIRST 30 DAYS! NO INVESTMENT Generate $50 to $100 whenever you have a couple of hours free time to spare. You could make $50 or more in the next 2 hours. Starting right Now!Today! http://snipurl.com/11i6lw GET PAID TO: Take online surveys and make from $5 to $75, or more Participate in focus groups and make up to $150 an hour Take phone surveys and you can earn as much as $120 an hour Try new products (and keep the free products too) Preview new movie trailers for $4 to $25 an hour http://snipurl.com/11i6lw Earn from your free website Awesome earnings get paid for your honest work Join as a free member and get paid to your bank account To join the Network follow the link http://snipurl.com/11i6lw -- http://mail.python.org/mailman/listinfo/python-list
Re: WMI in Python
Lawrence D'Oliveiro wrote: > Why not just call Scriptomatic directly from within the Python script, then? Because Scriptomatic _generates scripts to access WMI_, that's what it _does_. Are you _seriously_ advocating writing Python code to fire up a Windows application, programmatically manipulating a GUI to generate more Python code for your original script to import or exec? For your question to make any real sense, you'd want to ask "Why not access WMI directly...", which you might notice was the first recommendation I made. > Machine-generated code has no place in a source file to be maintained by a > human. As you've made your disdain for Windows _perfectly_ clear through your incessant sniping on this list, I think I'm pretty safe in saying you have no idea of the quality of Scriptomatic's output. I don't understand what possibly makes you believe you're qualified to make such a statement. -- http://mail.python.org/mailman/listinfo/python-list
Re: Argparse: add_argument with action=append appends to default
John O'Hagan wrote:
> I find this surprising:
>
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--test', action='append', default=['default'])
> [...]
parser.parse_args(['--test', 'arg'])
> Namespace(test=['default', 'arg'])
>
> As an argument is provided, I didn't expect to see the default in there as
> well. From the argparse docs: "the default value is used when the option
> string was not present at the command line". While it doesn't say "_only_
> when...", that's what I would have expected.
>
> Is there a reason for this behaviour? Is there a way to do what I want, to
> get a list of appended options _or_ a default list (apart from setting
> defaults outside the parser, a good way to lose track of things)?
You could either tweak the default value or the action:
$ cat append.py
import argparse
class DefaultList(list):
def __copy__(self):
return []
if __name__ == "__main__":
p = argparse.ArgumentParser()
p.add_argument("-t", "--test", action="append",
default=DefaultList(["default"]))
print p.parse_args()
$ python append.py -t one -t two
Namespace(test=['one', 'two'])
$ python append.py -t one
Namespace(test=['one'])
$ python append.py
Namespace(test=['default'])
$ cat append1.py
import argparse
class DefaultAppend(argparse._AppendAction):
def __call__(self, parser, namespace, values, option_string=None):
items = argparse._copy.copy(argparse._ensure_value(namespace,
self.dest, []))
try:
self._not_first
except AttributeError:
self._not_first = True
del items[:]
items.append(values)
setattr(namespace, self.dest, items)
if __name__ == "__main__":
p = argparse.ArgumentParser()
p.add_argument("-t", "--test", action=DefaultAppend,
default=["default"])
print p.parse_args()
$ python append1.py -t one -t two
Namespace(test=['one', 'two'])
$ python append1.py -t one
Namespace(test=['one'])
$ python append1.py
Namespace(test=['default'])
Peter
--
http://mail.python.org/mailman/listinfo/python-list
Re: select() call and filedescriptor out of range in select error
On Thu, 16 Sep 2010 15:51:38 +1000, James Mills wrote: > On Thu, Sep 16, 2010 at 2:49 PM, Ned Deily wrote: >> If the problem is that the socket object in question no longer exists, >> you can protect your code there by enclosing the remove operation in a >> try block, like: > > > The question that remains to be seen however is: > > Why does your list contain dirty data ? Your code has likely removed the > socket object from the list before, why is it attempting to remove it > again ? > > I would consider you re-look at your code's logic rather than patch up > the code with a "band-aid-solution". Well said. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
