[issue13285] signal module ignores external signal changes
Vilya Harvey vilya.har...@gmail.com added the comment: Petri: yes, that what I was suggesting. Charles-François: I'm certainly open to alternatives. Unless I've overlooked something though, the problem is that no workaround is possible at the moment. BTW this came up in the context of a customer script for the software I work on, so it's not just a theoretical problem - at least, not for me. :-) I guess another solution would be methods to save and restore the native signal handers, e.g. savesignal(signum) restoresignal(signum). That wouldn't entirely solve the problem - if someone called setsignal() before calling savesignal(), they'd end up in the same situation - but it would at least permit a workaround. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13285 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13285] signal module in ignores external signal changes
New submission from Vilya Harvey vilya.har...@gmail.com: The signal module is oblivious to any changes to the set of installed signal handlers which occur outside of the module. This can happen when a native module changes a signal handler, or when the python interpreter is embedded in another program which installs its own signal handlers. In this case, saving and restoring a signal handler through python doesn't work correctly. For example, if the SIGINT handler is set externally after the signal module is initialised, the following code will replace the external signal handler with python's default_int_handler: handler = signal.getsignal(signal.SIGINT) signal.signal(signal.SIGINT, handler) So it's impossible to reliably save and restore signal handlers through python when they can also be changed outside the python interpreter. Also, if there's a signal handler installed before the module is initialised, signal.getsignal() will return None for it - making it impossible to restore the correct handler after disabling it. The reason is that the signal module only checks for existing handlers when it's initialised. The results get stored in the Handlers array, which is then used by all subsequent calls to signal.getsignal(). There are no further checks to see whether the native signal handlers have changed. -- messages: 146553 nosy: vilya priority: normal severity: normal status: open title: signal module in ignores external signal changes type: behavior versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13285 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13285] signal module ignores external signal changes
Changes by Vilya Harvey vilya.har...@gmail.com: -- title: signal module in ignores external signal changes - signal module ignores external signal changes ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13285 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13285] signal module ignores external signal changes
Vilya Harvey vilya.har...@gmail.com added the comment:
Could it return an opaque wrapper object, rather than just the raw address?
Something like:
typedef struct _PyNativeSignalHandler {
PyObject_HEAD
sighandler_t handler_func;
} PyNativeSignalHandler;
where the type object doesn't expose any way to read or manipulate the
handler_func. Would that work, do you think?
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13285
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Best Way to Handle All Exceptions
2009/7/13 seldan24 selda...@gmail.com: Thank you both for your input. I want to make sure I get started on the right track. For this particular script, I should have included that I would take the exception contents, and pass those to the logging module. For this particular script, all exceptions are fatal and I would want them to be. I just wanted a way to catch them and log them prior to program termination. The logging module has a function specifically to handle this case: try: # do something except: logging.exception(Uh oh...) The exception() method will automatically add details of the exception to the log message it creates; as per the docs, you should only call it from within an exception handler. Hope that helps, Vil. -- http://mail.python.org/mailman/listinfo/python-list
Re: Memory error due to big input file
2009/7/13 Aaron Scott aaron.hildebra...@gmail.com: BTW, you should derive all your classes from something. If nothing else, use object. class textfile(object): Just out of curiousity... why is that? I've been coding in Python for a long time, and I never derive my base classes. What's the advantage to deriving them? class Foo: uses the old object model. class Foo(object): uses the new object model. See http://docs.python.org/reference/datamodel.html (specifically section 3.3) for details of the differences. Vil. -- http://mail.python.org/mailman/listinfo/python-list
Re: Question about generators
2009/7/12 Cameron Pulsford cameron.pulsf...@gmail.com: My question is, is it possible to combine those two loops? The primes generator I wrote finds all primes up to n, except for 2, 3 and 5, so I must check those explicitly. Is there anyway to concatenate the hard coded list of [2,3,5] and the generator I wrote so that I don't need two for loops that do the same thing? itertools.chain([2, 3, 5], primes) is what you're looking for, I think. Vil. -- http://mail.python.org/mailman/listinfo/python-list
Re: count
2009/7/8 Dhananjay dhananjay.c.jo...@gmail.com: I wanted to sort column 2 in assending order and I read whole file in array data and did the following: data.sort(key = lambda fields:(fields[2])) I have sorted column 2, however I want to count the numbers in the column 2. i.e. I want to know, for example, how many repeates of say '3' (first row, 2nd column in above data) are there in column 2. One thing: indexes in Python start from 0, so the second column has an index of 1 not 2. In other words, it should be data.sort(key = lambda fields: fields[1]) instead. With that out of the way, the following will print out a count of each unique item in the second column: from itertools import groupby for x, g in groupby([fields[1] for fields in data]): print x, len(tuple(g)) Hope that helps, Vil. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why is my code faster with append() in a loop than with a large list?
2009/7/6 Xavier Ho cont...@xavierho.com: Why is version B of the code faster than version A? (Only three lines different) Here's a guess: As the number you're testing gets larger, version A is creating very big list. I'm not sure exactly how much overhead each list entry has in python, but I guess it's at least 8 bytes: a 32-bit reference for each list entry, and 32 bits to hold the int value (assuming a 32-bit version of python). The solution you're looking for is a large 8 digit number; let's say 80,000,000, for the sake of easy calculation. That means, as you get close to the solution, you'll be trying to allocate almost 640 Mb of memory for every number you're checking. That's going to make the garbage collector work extremely hard. Also, depending on how much memory your computer has free, you'll probably start hitting virtual memory too, which will slow you down even further. Finally, the reduce step has to process all 80,000,000 elements which is clearly going to take a while. Version b creates a list which is only as long as the largest prime factor, so at worst the list size will be approx. sqrt(80,000,000), which is approx. 8900 elements or approx. 72 Kb or memory - a much more manageable size. Hope that helps, Vil. -- http://mail.python.org/mailman/listinfo/python-list
Re: finding most common elements between thousands of multiple arrays.
2009/7/4 Andre Engels andreeng...@gmail.com:
On Sat, Jul 4, 2009 at 9:33 AM, mclovinhanoo...@gmail.com wrote:
Currently I need to find the most common elements in thousands of
arrays within one large array (arround 2 million instances with ~70k
unique elements)
so I set up a dictionary to handle the counting so when I am
iterating I up the count on the corrosponding dictionary element. I
then iterate through the dictionary and find the 25 most common
elements.
the elements are initially held in a array within an array. so I am am
just trying to find the most common elements between all the arrays
contained in one large array.
my current code looks something like this:
d = {}
for arr in my_array:
-for i in arr:
#elements are numpy integers and thus are not accepted as dictionary
keys
---d[int(i)]=d.get(int(i),0)+1
then I filter things down. but with my algorithm that only takes about
1 sec so I dont need to show it here since that isnt the problem.
But there has to be something better. I have to do this many many
times and it seems silly to iterate through 2 million things just to
get 25. The element IDs are integers and are currently being held in
numpy arrays in a larger array. this ID is what makes up the key to
the dictionary.
It currently takes about 5 seconds to accomplish this with my current
algorithm.
So does anyone know the best solution or algorithm? I think the trick
lies in matrix intersections but I do not know.
There's no better algorithm for the general case. No method of
checking the matrices using less than 200-x look-ups will ensure
you that there's not a new value with x occurences lurking somewhere.
Try flattening the arrays into a single large array sorting it. Then
you can just iterate over the large array counting as you go; you only
ever have to insert into the dict once for each value and there's no
lookups in the dict. I don't know numpy, so there's probably a more
efficient way to write this, but this should show what I'm talking
about:
big_arr = sorted(reduce(list.__add__, my_array, []))
counts = {}
last_val = big_arr[0]
count = 0
for val in big_arr:
if val == last_val:
count += 1
else:
counts[last_val] = count
count = 0
last_val = val
counts[last_val] = count# to get the count for the last value.
If flattening the arrays isn't practical, you may still get some
improvements by sorting them individually and applying the same
principle to each of them:
counts = {}
for arr in my_array:
sorted_arr = sorted(arr)
last_val = sorted_arr[0]
count = 0
for val in sorted_arr:
if val == last_val:
count += 1
else:
counts[last_val] = counts.get(last_val, 0) + count
count = 0
last_val = val
counts[last_val] = counts.get(last_val, 0) + count
Hope that helps...
Vil.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Reversible Debugging
2009/7/4 Patrick Sabin patrick.just4...@gmail.com: If someone has another idea of taking a snapshot let me know. Using VMWare is not a very elegant way in my opinion. Someone implemented the same idea for Java a while ago. They called it omniscient debugging; you can find details at http://www.lambdacs.com/debugger/ and a paper about it at http://www.lambdacs.com/debugger/AADEBUG_Mar_03.pdf Another more recent paper on the topic is http://scg.unibe.ch/archive/papers/Lien08bBackInTimeDebugging.pdf I haven't read either of these papers myself, but maybe they'll give you some ideas. Vil. -- http://mail.python.org/mailman/listinfo/python-list
Re: finding most common elements between thousands of multiple arrays.
2009/7/4 Steven D'Aprano st...@remove-this-cybersource.com.au: On Sat, 04 Jul 2009 13:42:06 +, Steven D'Aprano wrote: On Sat, 04 Jul 2009 10:55:44 +0100, Vilya Harvey wrote: 2009/7/4 Andre Engels andreeng...@gmail.com: On Sat, Jul 4, 2009 at 9:33 AM, mclovinhanoo...@gmail.com wrote: Currently I need to find the most common elements in thousands of arrays within one large array (arround 2 million instances with ~70k unique elements) ... There's no better algorithm for the general case. No method of checking the matrices using less than 200-x look-ups will ensure you that there's not a new value with x occurences lurking somewhere. Try flattening the arrays into a single large array sorting it. Then you can just iterate over the large array counting as you go; you only ever have to insert into the dict once for each value and there's no lookups in the dict. You're suggesting to do a whole bunch of work copying 2,000,000 pointers into a single array, then a whole bunch of more work sorting that second array (which is O(N*log N) on average), and then finally iterate over the second array. Sure, that last step will on average involve fewer than O(N) steps, Er what? Ignore that last comment -- I don't know what I was thinking. You still have to iterate over all N elements, sorted or not. but to get to that point you've already done more work than just iterating over the array-of-arrays in the first place. What it does buy you though, as you pointed out, is reducing the number of explicit dict lookups and writes. However, dict lookups and writes are very fast, fast enough that they're used throughout Python. A line like: count += 1 actually is a dict lookup and write. I did some tests, just to be sure, and you're absolutely right: just creating the flattened list took several hundred (!) times as long as iterating through all the lists in place. Live and learn... Vil. -- http://mail.python.org/mailman/listinfo/python-list
[issue1791] The Library Reference still refers to the old bug tracker.
New submission from Vilya Harvey: The page at http://docs.python.org/lib/about.html refers people to the old SourceForge bug tracker, rather than bugs.python.org. -- messages: 59707 nosy: vilya severity: normal status: open title: The Library Reference still refers to the old bug tracker. versions: Python 2.5 __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1791 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1790] xmlrpclib ServerProxy page has out-of-date content
New submission from Vilya Harvey: The page at 'http://docs.python.org/lib/serverproxy-objects.html' contains the following text which should be removed as it's (a) not especially relevant; and (b) out of date: Introspection methods are currently supported by servers written in PHP, C and Microsoft .NET. Partial introspection support is included in recent updates to UserLand Frontier. Introspection support for Perl, Python and Java is available at the XML-RPC Hacks page. In particular: Python has built in introspection support now; and the URL for the XML-RPC hacks page returns a 404 error. -- components: Documentation messages: 59706 nosy: vilya severity: normal status: open title: xmlrpclib ServerProxy page has out-of-date content __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1790 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
